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DISCLAIMER 


This  Military  Operations  Research  Society  report  summarizes  the  proceedings  of  a  workshop 
conducted  over  three  days  by  experts,  users  and  participants  interested  in  quantifying  the 
relationship  between  testing  and  simulation.  It  is  not  intended  to  be  a  comprehensive  treatise 
on  the  subject.  It  reflects  the  major  concerns,  insights,  thoughts  and  directions  of  the 
participants  at  the  time  of  the  workshop. 

OSD  Disclaimer:  Review  of  this  material  does  not  imply  Department  of  Defense 
endorsement  of  factual  accuracy  or  opinion. 


CAVEATS 

■  The  Military  Operations  Research  Society  neither  makes  nor  advocates  official 
policy. 

■  Matters  discussed  or  statements  made  during  the  workshop  were  the  sole 
responsibility  of  the  participants  involved. 

■  The  Society  retains  all  rights  regarding  final  decisions  on  the  content  of  this 
workshop  report. 
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EXECUTIVE  SUMMARY 

As  Network  Centric  Operations  (NCO)  are  being  developed,  the  analytic  community  is  being 
called  upon  to  support  the  military  services  with  assessments  of  needs  and  capability  gaps  to 
support  decisions  concerning  options  under  consideration  by  the  United  States  Armed  Forces  and 
its  coalition  partners.  Many  organizations  are  interested  and  involved  in  employing  and 
analyzing  Network  Centric  Operations.  With  the  emergence  of  "Information  Age"  warfare,  the 
Operations  Analysis  (OA)  community  recognizes  it  must  play  a  leadership  role  in  creating  and 
refining  needed  metrics,  processes,  methodologies,  models  and  simulations.  The  OA 
Community  acknowledges  an  urgent  need  to  share  its  efforts,  successes  and  failures  in 
developing  the  key  capabilities  required  for  analytically  rigorous  assessments  of  NCO.  On  27-29 
January  2004,  MORS  held  a  workshop  on  Operations  Analysis  Support  to  Network  Centric 
Operations  at  the  Booz  Allen  Hamilton  Conference  Center  in  McLean,  VA.  The  goal  of  the 
meeting  was  to  bring  together  a  multi-disciplined  group  of  analysts,  operators  and  engineers 
fi'om  across  service  organizations  to:  share  their  work;  develop  a  common  view  of  the  state  of 
practice;  expose  members  of  the  broader  analytic  community  to  their  needs;  and,  identify 
shortfalls  and  recommend  solutions  for  improving  the  state  of  practice. 

The  mini-symposium  on  the  first  day  featured  presentations  calibrating  the  state  of  the  practice 
of  operations  analysis  support  to  NCO.  VADM  Arthur  Cebrowski,  USN  (Ret.),  Director,  Office 
of  Force  Transformation  provided  the  keynote  address  on  “Transforming  Defense  and 
Implementing  Network  Centric  Warfare.”  Next  the  Services  presented  their  perspective  with  an 
overview  of  the  challenges  and  progress  to  date  using  operations  analysis  to  support  NCO, 
including  highlighting  context  issues  and  what  is  needed  from  the  analytical  community,  and 
identifying  some  of  the  broad  and  difficult  analytical  questions  they  have  observed.  Finally,  the 
Technical  Co-Chairs  presented  briefs  to  frame  the  workshop  discussions  -  one  on  the  NCO 
definition  and  one  on  NCO  differences  among  countries. 

The  mini-symposium  was  followed  by  a  two-day  workshop.  The  participants  met  in  six  working 
groups:  1)  Measures  of  Merit',  2)  Processes  and  Methodologies',  3)  Assessment  Tools',  4)  NCO 
and  Force  Transformation',  5)  NCO  and  Coalition  Forces',  and,  6)  Applying  NCO  to  Actual 
Events.  A  Synthesis  Panel  was  formed  to  collect  and  summarize  insights  from  each  of  the 
working  groups. 

From  their  perspective,  the  working  groups  were  tasked  to  report  on  the  state  of  the  practice  of 
applying  operations  analysis  in  support  of  NCO;  identify  the  key  issues,  shortfalls  and  gaps; 
describe  the  future  challenges;  list  the  extent  of  collaboration  among  the  Services,  Coalition 
countries  and  United  States  Agencies;  and  develop  recommendations.  The  highlights  of  these 
reports  are  presented  in  this  section,  while  the  details  are  provided  in  the  “Workshop  Summary” 
section. 

For  the  state  of  the  practice  using  the  OA  Process,  the  measures  were  evaluated  as  frequently 
being  used,  but  inconsistent  in  their  application;  the  maturity  of  an  “NCO  OA  Process”  was 
considered  consistent  with  the  state  of  the  understanding  of  NCO  -  immature;  and  it  was 
concluded  that,  in  the  context  of  NCO,  the  tools  were  used  infrequently.  Current  measures,  tools 
and  practices  are  understandably  grounded  in  what  we  know  and  the  decision  process  we 
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support,  however,  applying  operations  analysis  in  the  area  of  NCO  is  in  its  infancy.  In  applying 
operations  analysis  to  NCO  concepts,  discussions  revealed  that  the  OA  process  is  currently 
limited  by  sparse  relevant  data,  the  real-world  operation  is  a  dynamic  environment  and  the  data  is 
not  clean  or  consistent.  Warfighting  systems  do  not  output  analytically-ready  data,  and  data 
collection  must  be  pre-planned  due  to  the  numerous  organizations  involved  and  their  focus  on 
real-world  operations. 

Network  Centric  Operations  can  be  characterized  as  possessing  four  key  domains  (physical, 
information,  cognitive  and  social).  In  general,  appl)dng  OA  across  the  four  domains  was 
considered  fair  or  poor:  physical  —  fair;  information  —  fair;  cognitive  -  poor;  and,  social  -  poor. 
Overall,  while  there  is  an  apparent  need  for  research  into  these  emerging  concepts,  there  is 
limited  evidence  that  the  OA  community  is  committed  to  expanding  its  analysis  into  the 
cognitive  and  social  domains.  Current  scientific  approaches  that  address  the  physical  domain 
well  only  address  a  subset  of  the  relevant  operational  space  and  apparently  do  not  represent  the 
other  domains. 

Highlights  of  the  working  group  findings  from  the  Synthesis  Group  were: 

WG 1:  Measures  of  Merit  (MOMs)  -  The  findings  focused  on  the  assessment  that  the  MOMs 
for  NCO,  especially  for  understanding  a  networked  “cause  and  effect”  relationship,  are  a  work- 
in-progress.  However,  previous  MOMs  for  the  physical  domains  for  warfighting  missions  are 
well  understood  and  remain  relevant. 

WG  2:  Processes  and  Methodologies  -  The  findings  focused  on  the  NCW  conceptual 
framework  and  the  gap  between  the  concept  of  sense-making  and  current  Military  Decision 
Making  Process  (MDMP).  It  is  also  suggested  that  a  multi-disciplinary  program  of  research 
should  be  initiated  to  explore  the  importance,  and  effective  representation,  of  the  cognitive  and 
social  domains  to  be  undertaken.  This  knowledge  should  then  be  used  as  a  foundation  for 
exploring  empirically  the  important  hypotheses  related  to  network  centric  operational  theory  and 
concepts. 

WG  3:  Assessment  Tools  -  The  findings  were  along  three  dimensions:  “better”  data; 
improvement  in  NCO  assessment  tools;  and,  the  networking  of  assessment  tools  in  order  to 
achieve  synergy  in  the  application  of  a  tool  set. 

WG  4:  NCO  and  Force  Transformation  -  To  effect  Force  Transformation  it  was  concluded 
that  the  further  we  move  away  from  assessing  transformations  that  are  focused  on  a  single 
Service,  tactics,  and  material  (“old”  missions,  etc.),  the  fewer  OA  tools  are  available  to  the 
analyst  for  information,  cognitive  and  social  domain  related  assessments. 

WG  5:  NCO  and  Coalition  Forces  -  It  was  discussed  that  the  OA  timelines  are  not  fast  enough 
to  keep  pace  with  technology  and  other  developments.  The  experience  of  the  UK’s  embedded 
OAs  needs  to  be  assessed  and  their  example,  maybe,  followed.  However,  the  inclusion  of 
civilian  agencies  also  must  be  addressed. 
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WG  6:  Applying  OA  to  Actual  NCO  Events  -  It  was  found,  because  policies  and  doctrine  to 
incorporate  analysis  is  lacking,  along  with  the  non-use  of  embedded  analysts,  that  OA  becomes 
an  afterthought.  If  these  findings  and  the  data  domain  are  improved,  then  the  assessment  of 
actual  NCO  events  will  be  improved. 

Based  on  the  issues,  findings,  shortfalls  and  gaps,  the  Synthesis  Group  developed  a  high-level  set 
of  actionable  recommendations  from  the  working  group  deliberations  and  discussions  with  the 
participants.  They  are: 

•  Add  a  MORS  Symposium  working  group  or  composite  group  dedicated  to  Network  Centric 
Operations.  NCO  is  much  more  than  C4ISR  and  Information  Operations  (10).  This  new 
working  group  or  composite  group  could  be  charged  to  examine  the  OA  toolkit  and  processes 
in  order  to  better  focus  on  the  NCO  problem.  At  the  72"*’  MORSS,  it  is  proposed  that  the 
CG  B  Chair  discuss  this  issue  with  the  leadership  from  Working  Groups  5-10,  providing 
recommendations  to  the  MORS  Working  Group/Composite  Group  Committee  Chair  and  the 
Vice  President  (Meeting  Operations). 

•  Create  a  center(s)  of  excellence  within  services  and  JFCOM  for  NCO  assessments  to  capture 
and  share  results  and  data  of  experiments,  training,  analysis  and  experience. 

•  Enable  (e.g.,  thru  agreements,  business  practices)  the  networking  of  analysts  working  NCO, 
and  provide  reach-back  for  tools,  data,  and  previous  studies. 

•  Establish  a  “business  model”  that  enables  services  to  conduct  operations  analysis  in  support 
of  “bom  joint”  NCO  concepts  and  that  is  underpinned  by  a  set  of  approved  tools  and  data. 

•  Conduct  follow-on  MORS  special  meetings  to  share  NCO  analysis  and  experimentation 
results.  Two  options  were  recommended: 

-  A  2-day  or  3-day  Mini-Symposium  to  educate  the  commimity  in  the  area  of  NCO 
analyses.  Training  and  education  was  seen  to  be  critically  important  to  transformation 
overall,  and  specifically,  to  the  operations  analysis  practice  in  support  of  NCO. 

-  A  second  3-day  Workshop  to  discuss  the  issues  more  fully.  It  was  suggested  that  we 
provide  foundation  presentations  during  the  morning  of  the  1®‘  day  and  outbriefs  during 
the  afternoon  of  the  3*^^  day  -  this  would  leave  two  full  days  for  discussion  and  further 
debate.  During  this  initial  workshop,  the  working  groups  felt  they  only  “scratched”  the 
surface  in  their  discussions. 

In  summary,  the  participants  believed  that  the  OA  process  developed  in  the  Industrial  Age  has 
continued  applicability  to  NCO  in  the  Information  Age.  We  are  only  beginning  to  understand 
the  practices  that  should  be  followed  to  perform  credible  NCO  analyses.  In  addition,  we  know 
there  are  NCO  considerations  that  remain  challenging.  However,  it  was  judged  that  we  do  not  do 
a  satisfactory  job  in  documenting,  sharing  and  learning  from  NCO  analyses  performed. 

With  the  arrival  of  terrorism,  other  asymmetric  threats,  rogue  nations  and  increased  concern  for 
homeland  defense,  NCO  presents  an  opportunity  for  Force  Transformation.  Although  these  are 
daunting  challenges,  if  the  recommendations  of  this  workshop  are  implemented  successfully, 
they  will  provide  a  basis  for  improving  the  operations  analysis  process  and  empowering  the  OA 
community  to  address  the  most  critical  analytical  challenges  to  support  NCO  and  Force 
Transformation. 
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WORKSHOP  SUMMARY 

BACKGROUND 

A  MORS  Workshop  on  Operations  Analysis  Support  to  Network  Centric  Operations  was  held  at 
the  Booz  Allen  Hamilton  Conference  Center  in  McLean,  Virginia,  on  27-29  January  2004.  1 52 
analysts  and  decision  makers  participated.  Of  these,  94  were  existing  or  former  members  and  58 
were  new  to  MORS  or  non-members.  Among  these  were  26  foreign  personnel  from  the  United 
Kingdom,  Canada,  Australia,  Netherlands,  Denmark,  Sweden  and  Israel. 

WORKSHOP  DESCRIPTION 

Mini-Symposium  Overview  (U‘  Day)  - 

The  mini-symposium  on  the  first  day  featured  papers  to  bring  us  up  to  speed  on  the  state 
of  the  practice  of  operations  analysis  support  to  NCO.  After  the  welcome  and 
introductions  by  the  MORS  President  (LTC  Willie  McFadden,  USA),  Host  (Mr.  Steve 
Stamer,  BAH)  and  General  Co-Chairs  (Mr.  Dennis  Baer,  Northrop  Grumman  IT  and  Mr. 
Kirk  Michealson,  Lockheed  Martin  Advanced  Concepts),  VADM  Arthur  Cebrowski, 
USN  (Ret.),  Director,  Office  of  Force  Transformation  provided  the  keynote  address  on 
“Transforming  Defense  and  Implementing  Network  Centric  Warfare.” 

Next  the  Services  were  invited  to  provide,  from  their  perspective,  an  overview  of  the 
challenges  and  progress  to  date  using  operations  analysis  to  support  NCO,  including 
highlighting  context,  issues  and  what  is  needed  from  the  analytical  community,  and 
identifying  some  of  the  broad  analytical  and  difficult  questions  they  have  observed.  The 
Service  presentations  were  provided  by  COL  Steven  Mains  (Joint  Forces  Command,  J9, 
Lessons  Learned  Group),  Mr.  Michael  Bauman,  FS  (Director  TRADOC  Analysis  Center, 
Fort  Leavenworth,  KS),  Dr.  Michael  Bell  (Senior  Analyst,  FORCEnet  Requirements 
Branch,  Office  of  the  CNO)  and  Dr.  George  Cran  (NEC  Programme  Leader,  Defence 
Science  and  Technology  Laboratory,  UK  Ministry  of  Defence). 

Dr.  Richard  Hayes,  President  of  Evidence  Based  Research,  Inc.  and  Workshop  Technical 
Co-Chair,  provided  the  Framework  Presentations.  The  first  offered  a  foundation  of 
Network  Centric  Operations  in  a  brief  titled  “C2  in  the  Information  Age:  The  Last  Mile 
of  Transformation,”  while  the  second  discussed  the  conceptual  differences  between  the 
United  States  and  Coalition  Countries  with  respect  to  NCO  and  Transformation. 

The  final  sessions  of  the  day  were  presentations  that  offered  guidance  for  the  remaining 
two  days  of  the  workshop.  Dr.  Jerry  Kotchka,  FS  (Lockheed  Martin  Integrated  Systems 
and  Solutions)  provided  a  review  of  the  operations  analysis  process,  Ms.  Sue  Iwanksi 
(Northrop  Grumman  IT)  followed  with  an  overview  of  the  MORS  Analyzing  Effects- 
Based  Operations  Workshop,  and  LtCol  Greg  McIntyre  (Air  Force  Studies  and  Analyses 
Agency  (APS  AA))  offered  an  overview  of  the  MORS  Operations  Research  Methods  for 
Information  Operations  Workshop. 
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Workshop  Overview  (2"^’  &  Days)  - 


The  mini-symposium  was  followed  by  a  two-day  workshop.  The  participants  met  in  six 
working  groups:  1)  Measures  of  Merit,  2)  Processes  and  Methodologies',  3)  Assessment 
Tools',  4)  NCO  and  Force  Transformation',  5)  NCO  and  Coalition  Forces',  and,  6) 
Applying  NCO  to  Actual  Events.  A  Synthesis  Group  was  formed  to  examine  insights 
across  all  six  working  groups  and  to  help  bring  together  a  consistent  set  of  workshop 
findings. 

A  listing  of  the  acronyms  used  in  this  final  report  is  provided  in  Appendix  A,  the  Terms 
of  Reference  for  this  special  meeting  is  contained  in  Appendix  B,  a  directory  of  the 
working  group  leadership,  an  energetic  group  of  people,  is  provided  in  Appendix  C,  and 
the  papers  presented  are  provided  in  Appendix  D. 

NCO  DEFINITION 

A  common  definition  of  Network  Centric  Operations  was  developed  by  the  Technical  Co-Chairs 
(Dr.  David  Alberts,  OSD  (Nil),  and  Dr.  RicWd  Hayes).  Network  Centric  Operations  (NCO) 
involves  the  development  and  employment  of  mission  capability  packages  that  are  the 
embodiment  of  the  tenets  of  Network  Centric  Warfare  (NCW)  in  operations  across  the  full 
mission  spectrum.  These  tenets  state  that  a  robustly  networked  force  improves  information 
sharing  and  collaboration,  which  enhances  the  quality  of  information,  the  quality  of  awareness, 
and  improves  shared  situational  awareness.  This  results  in  enhanced  collaboration  and  enables 
self-synchronization  improving  sustainability  and  increasing  the  speed  of  command,  all  of  which 
ultimately  result  in  dramatically  increased  mission  effectiveness. 

The  principles  of  Effects  Based  Operations  (EBO)  go  hand-in-hand  with  NCO  because  warfare, 
particularly  effective  warfare,  has  always  been  effects-based.  Sun  Tzu,  Genghis  Khan, 

Napoleon,  Eisenhower,  and  Schwarzkopf  all  would  be  familiar  with  the  principles  that  1) 
warfare  should  include  all  the  instruments  of  national  power;  and,  2)  that  each  instrument  should 
be  applied  in  a  way  that  maximizes  its  desirable  impact,  minimizes  undesirable  ones,  and 
complements  actions  taken  in  other  arenas.  These  basic  principles,  which  define  the  essence  of 
EBO,  occur  in  a  context  that  makes  them  particularly  relevant  today.  First,  we  have  the  means  to 
gather,  integrate  and  apply  more  data,  information  and  knowledge  than  analysts  and  policy 
makers  in  earlier  eras  -  we  are  in  the  Information  Age.  The  tenets  of  NCW  address  these  means 
and  postulate  how  they  can  increase  mission  effectiveness.  The  seven  tenets  of  NCW  are:  a 
robustly  networked  force;  information  sharing;  collaboration;  quality  of  information;  shared 
situational  awareness;  self-synchronization;  and,  sustainability  and  speed  of  command.  Second, 
we  live  in  a  world  that  is  more  tightly  coupled  than  ever  before,  creating  opportunities  and 
challenges  for  direct  and  indirect,  desirable  and  undesirable  effects. 

NCO  encompasses  Networked  Enabled  Capability  (NEC)  (UK,  Australia),  Networked-Based 
Defense  (Sweden),  and  concepts  firom  other  nations  also  based  upon  operationalizing  the  tenets 
ofNCW. 
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KEYNOTE  AND  SERVICE  PRESENTATIONS 


VADM  Arthur  Cebrowski,  Director,  Force  Transformation,  provided  the  keynote  address.  Some 
of  the  presentation  highlights  participants  could  consider  for  their  working  group  discussions 
included:  1)  speed  and  agility  should  be  achieved  over  optimization;  2)  the  warfighter  advantage 
exploits  behavioral  change  and  new  doctrine  to  enable  self-synchronization,  speed  of  command 
and  increased  combat  power;  3)  information  sharing  is  a  new  source  of  power;  and,  4)  measuring 
values,  attitudes  and  beliefs  should  be  considered.  During  his  presentation,  he  also  issued  a 
challenge  to  the  MORS  Community  -  to  create  tools  that  could  be  used  for  NCO  and 
transformation. 

After  the  Keynote  Presentation,  the  Services  provided  a  brief  overview  of  the  state  of  the  practice 
of  applying  operations  analysis  to  Network  Centric  Operations  from  their  perspective. 

COL  Steven  Mains  (JFCOM,  J9,  Deputy  Director,  Lessons  Learned  Group)  provided  the  Joint 
Staff  perception.  Some  of  the  highlights  from  his  presentation  were: 

•  Data  is  found  -  Information  is  made. 

•  Collection  does  not  equal  knowledge.  A  huge  “frame  rate”  could  be  counterproductive. 

•  Networks  do  not  produce  knowledge.  Networks  pass  data  between  people  that  can  turn  it 
into  information  and  information  between  people  that  can  turn  it  into  knowledge. 

•  Knowledge  is  the  key  to  Network-Centric  Operations 

Mr.  Michael  Bauman,  FS,  Director,  TRADOC  Analysis  Center,  presented  the  Army  point  of 
view  highlighting  the  analysis  completed  for  the  Future  Combat  System  (FCS)  including: 

•  The  FCS  program  and  focused  Army  investments  in  new  modeling  have  advanced  the 
Army’s  ability  to  model  and  analyze  network-enabled  operations  (aka  NCO). 

•  The  military  OA  community  still  faces  challenges: 

-  The  complexity  of  NCO  scenarios,  overwhelming  data  and  ambiguous  cause-and-effect 
relationships. 

-  Metrics  are  needed  that  are  useful  to  analysts  and  decision  makers,  but  that  also  resonate 
with  warfighters. 

•  The  “business  model”  to  enable  OA  in  support  of  joint  operations,  to  include  NCO,  is  still  an 
exploratory  work-in-progress. 

-  A  means  to  generate  data  for  joint  system  concepts  is  evolving. 

-  A  mechanism  to  achieve  authoritative  representation  of  future  joint  concepts  is  much  less 
mature. 

Dr.  Michael  Bell,  Senior  Analyst,  FORCEnet  Requirements,  N61,  followed  with  the  Navy’s 
status  with  their  NCO  system,  FORCEnet.  Some  highlights  were: 

•  Demonstrated  FORCEnet’s  value  of  analytical  framework: 

-  Connects  FORCEnet  capabilities  and  NCW  Framework. 

-  Capabilities  are  under  revision. 

-  Quantitative  measures  partially  successful  and  improvements  have  been  proposed. 
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Improved  representation  of  how  knowledge  is  achieved  in  C4ISR  campaign  analysis 


The  final  presentation  was  on  the  Coalition  perspective.  Dr.  George  Cran,  UK  MoD  Network 
Enabled  Capability  (NEC)  Programme  Leader,  presented  the  United  Kingdom  view.  Dr.  Cran 
highlighted  the  operations  analysis  issues  for  NEC: 

•  Bringing  together  future  military  thinking  with  future  technological  capabilities  across  the 
spectrum  of  operations  and  levels  of  command. 

•  Thinking  out  of  the  box  -  doing  better  things. 

•  Bringing  all  lines  of  development  into  the  analysis,  especially  people  issues. 

•  Bringing  practicalities  to  the  vision:  drawing  operational  lessons  learned  and  experimentation 
into  the  analysis. 

•  Enhancing  the  tool-set  to  reflect  NEC  with  greater  fidelity. 

•  Conducting  Balance  of  Investment  studies  between  ‘hard’  systems  and  information  systems. 
FRAMEWORK  PRESENTATIONS 

The  Technical  Co-Chairs,  Dr.  David  Alberts  and  Dr.  Richard  Hayes,  provided  two  framework 
discussions.  The  first,  titled  C2  in  the  Information  Age:  The  Last  Mile  of  Transformation,  was 
intended  to  provide  a  common  definition  of  Network  Centric  Operations  for  the  working  groups. 
Some  of  the  discussion  points  included  the  value  of  the  NCO  Metrics  Framework,  the 
prerequisites  for  transformation,  power  to  the  edge,  making  it  happen  —  becoming  an  “edge” 
organization,  and  the  OSD  (Nil)  initiatives  for  building,  populating  and  protecting  the  network. 
The  second,  titled  Network  Centric  Operations:  Differences  in  Perspective,  was  offered  to 
present  the  network  centric  differences  among  the  United  States,  United  Kingdom,  Sweden, 
Australia  and  NATO. 

RELEVANT  PREVIOUS  WORKSHOPS 

Representatives  from  two  of  the  relevant  previous  MORS  Workshops  provided  an  overview  of 
their  meeting’s  objectives  and  recommendations,  and  then  offered  the  participants  some  points  to 
consider  as  they  were  preparing  to  meet  in  their  working  groups. 

As  one  of  the  Technical  Co-Chairs  for  the  Analyzing  Effects-Based  Operations  Workshop, 

Ms.  Sue  Iwanski  passed  along  the  following  take-away  thoughts: 

•  The  term  “Indicators  of  Success”  was  used  for  EBO  since  it  expands  beyond  traditional 
Measures.  NCO  may  also  require  non-traditional  Measures. 

•  Fundamental  Sciences  were  considered  to  be  useful  for  EBO. 

•  The  EBO  tool  chest  was  envisioned  to  include  easily  manipulated,  specialized  modeling  and 
simulation  tools,  computational  social  science  tools,  data  mining,  colored  Petri  nets,  neural 
networks,  and  specialized  tools  developed  in  particular  application  areas. 

•  Networks  are  targets  in  EBO.  Information  exchange  is  vital  to  NCO. 

As  the  Chair  for  the  Operations  Research  Methods  for  Information  Operations  Workshop,  Dr. 
Richard  Deckro  prepared  a  list  of  points  to  consider  for  the  participants: 
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•  Build  MOEs  and  Battle  Damage  Assessments  for  NCO 

-  Extend  into  EBO  MOEs  for  NCO  use 

-  What  should  a  NCO  “JMEMS”  look  like? 

-  Does  the  NCO  framework  help? 

•  What  do  Commanders  need  to  use  NCO  as  a  “primary  arrow”  in  their  quiver? 

-  Leadership  understanding 

-  Tools 

-  Intelligence 

•  What  are  the  needs  of  the  human  element? 

GOALS  AND  OBJECTIVES 

Many  organizations  are  attempting  to  analyze,  understand,  and  employ  Network  Centric 
Operations.  The  analytic  community  needs  to  support  the  military  services,  joint  community, 
and  other  stakeholders  with  assessments  as  NCO  is  being  developed.  The  community  should 
play  a  leadership  role  in  creating  and  refining  the  metrics,  processes,  methodologies,  models  and 
simulations  necessary  to  understand  the  emerging  area.  The  community  should  share  efforts, 
successes  and  failures  in  the  key  capabilities.  As  a  first  step,  the  goal  of  the  meeting  was  to 
provide  an  opportunity  to  bring  a  multi-disciplined  team  of  analysts,  operators  and  engineers 
from  those  organizations  together  to  share  their  work,  develop  a  common  view  of  the  state  of 
practice,  expose  members  of  the  broader  analytic  community  to  their  needs,  identify  shortfalls 
and  potential  solutions. 

The  overall  objectives  of  the  workshop  were  to  provide  an  assessment  and  a  roadmap  to 
revitalize  the  state  of  the  analytical  practice  as  it  relates  to  NCO,  and  to  recommend  priorities  for 
any  initiatives  identified.  In  other  words,  to  assess: 

•  “State  of  Health”  -  Provide  an  assessment  of  the  state  of  analytical  practice  related  to  NCO. 

•  “Findings”  -  Provide  a  roadmap  to  revitalize  that  state. 

•  “Recommendations”  -  Recommend  priorities  for  any  initiatives  identified  in  the  roadmap. 
Some  specific  objectives  for  the  Working  Groups  were  as  follows: 

•  WG 1:  Measures  of  Merit.  A  great  deal  of  work  has  gone  into  developing  a  conceptual 
framework  for  Network  Centric  Warfare.  An  extensive  body  of  literature  exists  and  robust 
discussion  continues.  Measures  of  merit  have  been  addressed  often  in  this  context.  Several 
approaches  have  been  proposed  for  measuring  the  effectiveness  of  Network  Centric 
Operations.  Much  of  the  discussion  has  focused  on  measures  of  performance  for  the  network 
that  enables  NCO.  One  reason  for  this  inclination  is  that,  relative  to  other  areas  of  NCO 
theory,  data  is  readily  available.  However,  NCW  is  not  simply  a  new  communications 
system  with  embedded  decision  support.  Therefore,  it  is  important  to  examine  the  impact  of 
NCW  technologies  and  practices  across  the  physical,  information,  cognitive  and  social 
domains.  To  devise  metrics  for  these  other  aspects  of  NCO,  several  approaches  were 
considered  for  the  workshop.  The  Network  Centric  Operations  Conceptual  Framework 
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(Figure  1)  was  the  starting  place.  Discussions  on  cognitive  and  social  aspects  also  were 
reviewed  to  ensure  that  the  appropriate  measures  and  metrics  were  considered. 

Figure  1  -  NCO  Conceptual  Framework 


•  fFG2:  Processes  and  Methodologies.  There  were  three  objectives:  (1)  use  the  NCO 
Conceptual  Framework  to  assess  operations  analysis  ability  to  evaluate  NCO  enabling 
capabilities,  (2)  identify  areas  of  weakness  in  both  the  NCO  Conceptual  Framework  and  the 
OA  Process,  and  (3)  recommend  near  and  longer  term  actions  that  will  improve  the 
operations  analysis  community  to  support  NCO  related  research,  analyses,  and 
implementation. 

•  WG  3:  Assessment  Tools.  The  objective  of  this  working  group  was  to  assess  various  models 
across  the  four  Network  Centric  Warfare  domains.  Models  are  a  set  of  algorithms  and  data, 
with  supporting  infrastructure  that  manages  output,  random  number  generation,  etc.  For  each 
major  algorithm  that  models  an  important  aspect  of  NCO,  whether  contained  in  the  existing 
tools  or  those  imder  development,  the  modeling  process  was  examined  to  determine  if  it  is 
mature,  developing,  immature  or  unknown.  The  working  group  also  discussed  if  research  is 
needed  to  increase  understanding  of  how  specific  NCO  aspects  can  be  assessed  or  new 
algorithms  developed.  For  data,  an  insight  gained  from  a  model  is  the  combination  of  the 
data  and  the  model’s  transformations  of  input  data  into  output.  Given  all  of  the  existing  and 
proposed  tools  that  may  be  used  for  NCO,  the  state  of  the  data  required  to  make  these  tools 
“study  ready”  was  assessed. 
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•  IVG  4:  NCO  and  the  Force  Transformation  Process.  Working  Group  4  examined  how 
operations  research  methods  can  be  applied  to  NCO  in  the  context  of  the  Force 
Transformation  Process.  Transformation  will  only  happen  when  the  eight  “key  elements” 
(concepts,  leadership,  education,  culture,  training,  organization,  process  and  technology)  co¬ 
evolve  to  take  full  advantage  of  information  age  technologies,  capabilities,  and  opportunities. 
The  key  task  for  the  Force  Transformation  group  was  to  understand  how  OA  techniques  can 
be  applied  across  the  eight  “key  elements”  of  transformation  and  balance  the  transformation 
process. 

•  WG  5:  NCO  and  Coalition  Forces.  There  were  three  major  objectives  of  this  working 
group:  (1)  to  review  and  discuss  papers  that  address  current  practice  in  the  areas  of  tools, 
studies,  and  experimentation  or  live  events,  (2)  to  discuss  each  individual  country’s  state  of 
practice  of  OA  in  support  of  NCO,  to  identify  good  and  bad  points  and  to  identify  future 
directions,  and  (3)  to  assess  collaborative  efforts  among  coalition  partners  to  determine 
shortcomings  and  to  recommend  ways  to  improve  collaborative  OA  efforts.  Papers  were 
presented  in  three  groups:  (1)  country  study  overviews,  (2)  tools,  and  (3)  experiments  and 
live  events. 

•  WG  6:  Applying  NCO  to  an  Actual  Event.  This  working  group  examined  the  application 
of  Network  Centric  Operations  in  support  of  recent  operations  in  Afghanistan  and  Iraq.  The 
working  group  reviewed  the  data  for  Operations  Desert  Storm  and  Iraqi  Freedom  to  look  for 
the  limiting  factors,  the  tools  and  techniques,  and  the  different  ways  that  operations  analysis 
can  help  take  information  and  enable  events. 

BROAD  OBSERVATIONS 

A  few  broad  observations  of  the  workshop  were  noted.  First,  there  exists  a  need  to  include 
service  senior  analytical  leaders  and  experienced  analysts  in  MORS  workshops  to  enhance 
interactions,  findings,  recommendations,  and  to  mentor  more  junior  participants.  Second,  a  joint 
business  model  is  needed  that  enables  services  and  joint  commands  to  collaboratively  conduct 
operations  analysis  in  support  of  emerging  NCO  concepts.  This  will  facilitate  the  initiation  of 
OA  activity  early  in  the  plaiming  process.  Third,  networking  of  operations  analysis  centers  and 
operation  analysts  along  with  adequate  reach-back  should  provide  a  significant  improvement  in 
support  of  decision  makers.  An  environment  of  trust  is  needed  among  analysts  and  between  the 
analyst  and  decision  maker,  however.  Finally,  NCO  analysis  will  not  be  addressed  by  a  single 
measure  of  merit  (MOM)  but  by  a  set  of  MOMs. 

STATE  OF  THE  PRACTICE 

The  working  groups  were  asked,  if  possible,  to  provide  their  “State  of  Health”  in  two  contexts: 
the  Operations  Analysis  Process  (Figure  2)  and  the  Network  Centric  Warfare  four  key  domains 
(Figure  3).  Additionally,  the  seven  tenets  of  NCO  in  relation  to  the  framework  were  evaluated 
by  the  Processes  and  Methodologies  Working  Group  and  the  eight  key  elements  of 
transformation  were  considered  by  the  Force  Transformation  Working  Group. 


11 


Figure  2  -  Operations  Analysis  Process 


Within  the  OA  Process,  the  measures  were  evaluated  as  frequently  being  used,  but  inconsistent 
in  their  application;  the  maturity  of  an  “NCO  OA  Process”  was  considered  consistent  with  the 
state  of  the  understanding  of  NCO. .  ..immature;  and  it  was  the  conclusion  that,  in  the  context  of 
NCO,  the  tools  were  used  infrequently.  Current  measures,  tools  and  practices  are  understandably 
grounded  in  what  we  know  and  the  decision  process  we  support,  but  applying  operations  analysis 
in  the  area  of  Network  Centric  Operations  is  in  its  infancy.  The  United  States  Army’s  Future 
Combat  System  and  the  Navy’s  FORCEnet  activities  are  initial  efforts. 

In  applying  operations  analysis  to  NCO  events,  discussions  highlighted  the  perceptions  that:  the 
OA  process  is  currently  limited  by  available  and  valid  data,  the  real-world  operation  is  a  dynamic 
environment  and  the  data  is  not  clean  or  consistent,  warfighting  systems  do  not  output 
analytically-ready  data,  and  data  collection  must  be  pre-planned  due  to  the  numerous 
organizations  involved  and  their  focus  on  the  real-world  operation. 

Network  Centric  Operations  can  be  represented  using  four  key  domains  shown  in  Figure  3 
(physical,  information,  cognitive,  and  social).  The  state  of  health  for  OA,  in  relation  to  these 
domains,  was  estimated  as  deficient  or  poor.  Even  at  the  physical  level  of  modeling  and 
analysis,  it  appears  that  the  Operations  Analysis  community  continues  to  focus  on  the  warfare 
(combat)  portion  of  NCO  and  not  on  the  larger  set  of  military  missions  (e.g.,  OOTW)  -  as 
observed  in  the  scarcity  of  methods  for  these  situations.  In  general,  applying  OA  across  the  foiu 
domains  was  considered  fair  or  poor:  physical  -  fair;  information  -  fair;  cognitive  -  poor;  and 
social  -  poor.  Overall,  the  OA  community  seems  to  be  uncomfortable  with  expanding  its 
analysis  into  the  cognitive  and  social  levels.  Scientific  approaches  of  the  past  that  addressed  the 
physical  domain  well  apparently  do  not  represent  the  other  domains  as  well,  and  usually  are 
limited  in  focus  to  key  physical  interactions  associated  with  combat  and  combat  support 
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operations.  Methods  from  other  disciplines,  such  as  medicine,  genetics,  sociology,  psychology, 
political  science,  and  cultural  anthropology  fields,  should  be  considered  as  potential  sources  of 
knowledge  that  could  inform  the  OA  process  and  commimity. 

Figure  3  -  NCW  Four  Key  Domains 


Phyrfcal  Domain 

where  strike,  protect,  and  maneuver  take  place  across  different 

environments 


’  ;  ,  1 ,  ;  Information  D 

where  information  is  created,  manipulated  and  shared 


where  perceptions,  awareness,  beliefs,  and  values  reside  and  where;. 
:;as  a  res^^  decisions  afe  made';; 


Additionally,  the  seven  tenets  of  Network  Centric  Operations  in  relation  to  the  framework  were 
appraised  to  distinguish  what  OA  capabilities  can  be  done  reasonably  well  from  among  those  OA 


capabilities  that  are  consistently  deficient: 

• 

Robustly  networked  force 

Fair 

• 

Information  sharing 

Fair 

• 

Collaboration 

Poor 

• 

Quality  of  information 

Good 

• 

Shared  situational  awareness 

Fair 

Self-synchronization 

Poor 

• 

Sustainability  and  speed  of  command 

Poor 

The  major  discussion  points  on  these  seven  tenets  dealt  with  information  sharing,  collaboration 
and  cultural  issues.  For  information  sharing,  the  community  has  ideas  today  on  how  to  represent 
and  analyze  the  movement  of  information,  but  the  issues  associated  with  dynamic  networks  and 
Information  Assurance  caused  the  evaluation  of  the  “state  of  the  practice”  to  be  only  “fair”.  It 
also  was  felt  that  understanding  and  modeling  of  collaboration,  especially  at  the  social  level,  was 
very  weak.  Finally,  cross-cultural  issues  (i.e.,  across  agencies,  NGOs,  etc.)  could  benefit  from 
“technology  transfer”  from  other  fields  of  research. 
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Transformation  is  clearly  about  moving  from  an  industrial  age  force  to  an  information  age  force. 
The  Force  Transformation  working  group  took  the  view  that  transformation  will  only  happen 
when  the  eight  “key  elements”  (Figure  4)  co-evolve  to  take  full  advantage  of  information  age 
technologies,  capabilities,  and  opportunities.  The  key  task  for  the  working  group  was  to 
understand  how  OA  techniques  can  be  applied  across  the  eight  “key  elements”  of  transformation 
and  balance  the  transformation  process. 

Figure  4  -  Transformation  ...  Co-Evolution  of  Key  Elements 


Today 


Future 


The  operations  analysis  community  is  marginally  engaged  in  current  short-term  transformation 
efforts.  Short-term  transformation  is  occurring  through  operational  units  buying/using 
information  technology  systems  and  through  the  prototyping  efforts  of  the  Concept  Development 
and  Experimentation  (CD&E)  program.  Analysts  are  engaged  in  the  “lessons  learned”  teams 
deployed  to  Operations  Enduring  Freedom  (OEF)  and  Operation  Iraqi  Freedom  {OIF)  and  this 
is  encouraging.  However,  analysts  are  not  involved  early  enough  in  the  CD&E  process. 
Frequently,  the  analyst  is  not  called  upon  until  a  concept  is  ready  to  enter  the  experimentation 
phase  of  development.  Feeding  back  the  lessons  of  recent  operations  into  the  design  of  standing 
collection  plans  and  analysis  techniques  for  future  operations  would  create  an  environment  of 
continuous  learning. 

There  is  an  overabundance  of  (sometimes  conflicting)  guidance  governing  US  DoD 
transformation  efforts.  Some  of  this  guidance  is  informed  by  analysis,  but  additional  analysis 
will  be  critical.  Conflicting  guidance  makes  it  tough  to  develop  an  innovative,  effective 
transformation  program.  The  transformation  of  how  the  US  DoD  does  business  is  ongoing  and 
will  provide  the  foundation  and  governance  for  realizing  NCO.  This  will  be  a  precursor  to  any 
significant  long-term  deliberate  force  transformation  being  achieved.  Moreover,  Service  efforts 
are  not  linked.  While  they  are  not  being  conducted  in  isolation,  each  would  benefit  from  greater 
collaboration. 


The  current  budgeting  feedback  system  (Joint  Requirements  Oversight  Council  (JROC) 

Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel  and  Facilities  (DOTMLPF)) 
does  not  foster  transformation.  Some  may  argue  that  this  system  is  actively  hostile  to  the  notion 
of  trading  off  platform-centric  programs  for  network-centric  programs. 

The  analysis  community  has  not  been  proactively  engaged  in  the  transformation  process.  We  do 
have  some  analytically-derived  principles  (primarily  from  the  Command  and  Control  Research 
Program  (CCRP)  body  of  work),  but  we  need  to  actively  translate  these  principles  into  sound 
analysis  practice.  How  do  we  integrate  across  the  eight  disparate  elements?  This  requires  us  to 
fundamentally  examine  how  we  are  currently  conducting  analysis. 

Over  five  years  ago  at  the  MORS  Mini-Symposium  and  Workshop  on  “Analyzing  C4ISR  for 
2010,”  a  template  for  the  NATO  Code  of  Best  Practice  for  Assessing  Command  and  Control  was 
developed  and  evaluated.  This  template  was  used  to  evaluate  the  OA  process  for  NCO  and  the 
Synthesis  Group’s  overall  assessment  is  contained  in  Figure  5.  There  seems  to  be  an  overall 
decrease  in  the  adequacy  of  the  process,  due  mainly  to  a  feeling  that  NCO  as  a  whole  may  be 
more  difficult  to  evaluate  then  C4ISR  alone. 

Figure  5  -  Overall  Assessment  of  Applying  OA  to  NCO 


Legend: 
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With  inputs  from  each  of  the  working  groups,  the  Synthesis  Group’s  overall  qualitative 
assessment  across  the  NCW  four  key  domains  is  provided  in  Figure  6.  The  physical  domain  is 
assessed  to  be  in  much  better  “health”  from  the  application  of  the  OA  process  than  the  other 
three  domains. 

KEY  ISSUES 

The  participants  were  asked  to  consider  the  key  issues  for  each  of  their  working  group  areas. 
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For  metrics,  a  catalog  is  needed  to  enhance  OA  support  to  NCO.  There  are  several  ways  to 
organize  NCO  measures  that  make  intuitive  sense  but  examples  of  specific  measures  within 
these  categories  are  hard  to  find.  “Traditional”  measures  are  necessary  but  not  sufficient  to 
address  the  needs  of  analysts  in  the  NCO  area.  Measures  of  attrition  do  not  apply  to  all 
situations  in  NCO  analysis.  In  general,  qualitative  measures  should  be  accepted  for  some 
purposes.  Currently  accepted  metrics  generally  focus  on  degrees  of  achievement  of  desirable 
properties  related  to  systems.  A  more  holistic  set  of  metrics  that  address  unfavorable 
developments,  which  may  not  be  associated  with  specific  systems,  could  also  be  useful. 

Figure  6  -  Overall  Assessment  of  the  NCW  Four  Key  Domains 
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Cognitive  Domain 

where  perceptions,  awareness,  beliefs,  and  values  reside 
and  where,  as  a  result  of  sensemaking,  decisions  are  made 


Social  Domain 

where  force  entities  interact 
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Measuring  NCO  “effectiveness”  may  require  generation  of  non-traditional  MOEs  to  evaluate 
changes  in  performance  related  to  the  cognitive  and  social  domains.  However,  traditional 
warfighting  MOEs  remain  applicable  as  we  use  NCO  to  do  the  same  missions  differently.  A 
major  consideration  is  to  initiate  the  development  of  NCO  measures  of  merit  as  new  NCO 
concepts  and  experiments  are  developed. 

In  the  area  of  operations  analysis  processes  and  methodologies.  System  of  Systems  (SoS) 
architecture  analysis  may  be  pivotal  to  assessing  Network  Centric  Operations.  Yet,  the  classical 
OA  pyramid  of  modeling  &  simulation,  the  disciplined  OA  process,  and  even  the  detailed 
process  described  in  the  recent  book  NATO  Code  of  Best  Practices  for  Assessment  (DoD  CCRP, 
2002)  do  not  explicitly  address  the  role  of  architecture  related  analyses  in  the  assessment  of  SoS. 
This  lack  of  process  clarity  is  evident  as  the  OA  community  is  challenged  more  and  more  with 
complex,  SoS  solutions  (e.g.,  DOTMLPF)  for  meeting  the  capability  needs  for  NCO. 
Furthermore,  since  the  NCO  framework  emphasizes  “Quality  of  Individual  Sense-making”  and 
“Quality  of  Shared  Sense-making”  as  the  major  processes  in  the  cognitive  domain,  the  lack  of 
processes  and  methodologies  for  operations  analysis  using  sense-making  and  cognitive  modeling 


make  the  NCO  framework  a  challenge  to  operationalize.  A  major  issue  was  the  gap  between 
sense-making  and  decision  making,  along  with  a  lack  of  knowledge  as  to  the  “state  of  the 
analysis”  for  sense-making  and  cognitive  modeling. 

Tools  to  support  the  assessment  of  Network  Centric  Operations  must  address  a  wider  set  of 
outputs  than  attrition,  and  as  a  result,  traditional  combat  modellers  are  unlikely  to  have 
understanding  of  the  issues  required  to  model  NCO  properly.  Discussions  led  to  the  revelation 
that  there  are  few  tools  to  examine  the  relationship  between  the  quality  of  the  decisions  and  the 
amount  of  data  that  is  available  to  the  decision  maker.  Transforming  a  force  to  Network  Centric 
Operations  raises  serious  concerns  about  the  force’s  reliance  on  the  network.  Tools  need  to  be 
developed  to  assess  the  vulnerabilities  related  to  this  reliance  on  the  network.  Also,  if  coalition 
operations  are  to  be  undertaken  in  the  future  then  (at  least)  campaign  level  models  must  be  able 
to  capture  the  issues  related  to  information  flows  among  members  of  the  allied  forces.  This  is 
especially  important  as  forces  have  different  levels  of  ‘network  compliance’  in  both  their 
hardware  and  training/doctrine. 

Model/tool  development  projects  also  must  include  access  to  relevant  expertise  in  the  social  and 
cognitive  domains.  The  term  Network  Centric  Operations  is  sometimes  misleading.  At  a  closer 
look,  it  is  really  Human  Centric  and  Network  Enabled.  Addressing  the  Human  issues  is 
necessary  to  conduct  proper  assessment  of  NCO. 

The  “data  domain”  (both  real  blue  and  red  data  -  current  and  projected)  is  still  too  stove-piped. 
Each  working  group  assessed  that  the  data  domain  remains  an  area  for  significant  improvement. 
The  need  for  “standard,  approved”  data  for  friendly,  unfriendly  and  neutral  forces  -  both  current 
and  projected  -  would  be  a  major  enrichment.  Overall,  the  assessment  tools  for  NCO  were 
judged  to  need  improvement  -  the  OA  tools  for  the  physical  domain  are  more  adequate  then  the 
information,  cognitive  or  social  domains. 

In  the  area  of  Force  Transformation,  a  specific  example  of  the  analyst  being  on  the  sideline  is  in 
the  joint  experimentation  community  where  the  analyst  is  not  brought  in  until  relatively  late  in 
the  experiment  design  phase.  This  often  can  lead  to  an  experimental  design  that  does  not  support 
the  questions  of  interest.  This  type  of  behavior  is  symptomatic  of  the  broader  problem  of  project 
sponsors  not  being  engaged  with  the  analysis  community  in  some  circumstances.  The  result  can 
be  a  lack  of  ownership  of  the  results  of  the  analysis  performed  in  support  of  the  study.  The  next 
step  to  integrating  analysts  into  concept  development  and  experimentation  (CD&E)  is  the  design 
of  experiment  campaigns.  An  analyst’s  experience,  critical  thinking,  understanding  of  available 
models  and  measures  of  merit  can  be  a  valuable  resource.  The  analyst  can  assist  in  the  design  of 
objectives,  data  collection  and  analysis  techniques  that  assist  in  the  analysis  of  individual  events 
as  well  as  analysis  between  events. 

As  the  analytical  focus  of  NCW  moves  from  a  tactical  level  to  operational  and  strategic  levels, 
the  nature  of  the  system  components  and  interactions  under  investigation  becomes  more  complex 
because  of  the  increasing  focus  on  interactions  between  humans  instead  of  technology.  In 
general,  there  is  a  shift  from  quantitative  objective  measurements  to  qualitative  subjective 
measurements.  Traditional  OA  tools  do  not  support  the  upper  end  of  the  spectrum  well.  The 
representation  of  complex  adaptive  systems  across  the  four  domains  of  NCO  and  the  eight 
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elements  of  transformation  can  be  supported  by  the  softer  system  techniques.  The  operations 
analysis  community  should  look  to  other  disciplines  that  have  developed  techniques  for  dealing 
with  the  complexity  of  systems  centered  on  the  human. 

Trade-off  studies  between  material  solutions  have  traditionally  been  a  rich  area  for  OA 
techniques.  However  with  the  emerging  need  to  trade-off  between  platforms,  networks, 
information,  and  other  resource  investments,  a  set  of  common  metrics  that  can  serve  as  a  basis 
for  comparison  is  always  difficult  to  determine.  Currently,  studies  can  provide  at  best  an 
assessment  of  risk  associated  with  each  option,  but  not  definitive  options  for  a  decision  maker  to 
consider.  Thus  there  is  still  a  large  component  of  “intuition”  in  the  investment  decisions  being 
made. 

Collaboration  among  Coalition  nations  needs  to  be  improved.  Although  discussions  within  this 
Workshop  identified  previous  significant  involvement  and  sharing  of  allied  countries’  operations 
analysis  capabilities  to  support  NCO,  it  was  judged  that  collaboration  among  Coalition  nations 
remains  insufficient  due  to  multi  level  security,  OA  resources,  and  other  factors.  Similar 
constraints  (e.g.,  security)  will  also  impact  information  sharing  among  analysts. 

In  applying  OA  to  an  actual  NCO  event,  it  was  brought  out  that  the  Center  for  Army  Analyses 
(CAA)  had  to  overcome  some  inertia  to  get  its  analysts  accepted  into  the  combatant  command. 
Even  with  analysts  in  the  field,  typical  staff  officers  and  leaders  are  unfamiliar  with  the  tools  and 
capabilities  brought  by  a  trained  analyst.  UK  analysts  were  well  embedded  into  deployed  UK 
forces  and  have  a  history  of  being  so.  That  model  for  employment  of  analysts  could  serve  as  an 
example  for  the  US  program. 

Additionally,  the  deployed  OA  assets  were  not  aware  of  each  other’s  efforts.  Teams  went  into 
theater,  often  on  short  notice,  and  did  not  have  time  to  contact  other  agencies  prior  to  deployment 
to  become  familiar  with  and  coordinate  efforts.  Lack  of  collaboration  between  teams  (in  some 
cases)  prevented  sharing  techniques  and  data. 

As  recommended  in  the  MORS  Workshop  on  the  “Combat  Analyst,”  there  is  a  need  to 
consistently  deploy  and  network  analysts  with  the  operational  forces.  Information  sharing  will 
get  more  difficult  due  to  increasing  reliance  on  SIPRNET  and  similar  activities  and  will 
constrain  collaboration  among  analysts  as  well  as  operators.  The  deployment  and  networking  of 
operations  analysts  with  sufficient  reach-back  capabilities  is  a  major  issue  that  should  be 
addressed  as  NCO  concepts  and  tactics  are  developed. 

SHORTFALLS  AND  GAPS 

After  discussing  the  key  issues,  the  working  groups  were  asked  to  consider  their  shortfalls  and 
gaps  in  applying  OA  to  support  NCO. 

Most  metrics  in  current  use  for  NCO  fall  into  the  physical  and  information  domains.  This  is  not 
to  say  that  existing  measures  in  these  domains  fill  all  the  needs  there,  but  there  is  more  choice 
(and  understanding)  in  these  two  areas  than  in  the  cognitive  and  social  domains.  Additionally, 
useful  measures  are  constantly  being  developed  for  analytical  efforts  but  they  are  not  widely 
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disseminated  across  the  OA  commimity.  This  is  due  in  part  to  the  fact  that  most  analysis  is 
conducted  on  a  bilateral  basis,  at  best.  More  multilateral  projects  among  the  Services  and  with 
Coalition  partners  would  help.  Joint  staff  and  JFCOM  studies,  while  joint  in  nature,  do  not 
necessarily  involve  the  Services  and  Allies  sufficiently  to  achieve  cross-pollination  across  the 
OA  community. 

The  current  tools  and  practices  do  not  address  or  capture  collaboration,  self-synchronization,  or 
agility  -  key  characteristics  of  NCO.  Furthermore,  a  shortfall  in  tool  creation  and  application  is 
data  availability.  Data  for  creating/populating  models  is  difficult  to  acquire.  The  sources, 
classification,  accessibility,  and  releaseability  issues  hinder  the  productiveness  of  the  tools. 
Also,  results  that  are  classified  are  not  as  beneficial  to  the  community  as  a  source  of  common 
information  than  those  which  are  more  easily  distributed. 

Data  needs  to  be  gathered,  collected,  and  analyzed  by  social  scientists  working  collaboratively 
with  the  traditional  military  OA  commimity.  NCO  is  Human  Centric,  and  those  who  study 
humans  need  to  assist  in  gathering  the  data  because  they  may  recognize  behaviors  and 
characteristics  that  engineers  and  analysts  may  not  recognize.  The  human  dimensions  of  NCO 
are  not  currently  adequately  captured  and  explored. 

Analysts  need  to  engage  early  in  transformation  initiatives  to  structure  the  research  issues,  help 
define  the  question,  and  assess  the  value-added  of  experimentation  efforts.  Applying  the  OR 
method  and  current  tools  will  add  great  value.  Involvement  of  the  decision  maker  in  the  OA 
process  creates  ownership  of  analysis  and  starts  with  formulation  of  the  question  by  the  sponsor 
with  analyst  support. 

Current  OA  methods  do  not  lend  themselves  to  studies  of  complex,  adaptive  systems  that  cross 
the  four  key  NCW  domains  (problems  that  are  at  the  core  of  defense  transformation).  Legacy 
simulation  tools  are  not  especially  helpful  in  this  area.  Also,  there  is  not  a  coordinated  effort  to 
integrate  across/co-evolve  the  eight  key  elements  of  transformation  (concepts,  leadership, 
education,  culture,  technology,  organization,  process,  training). 

Several  shortfalls  in  the  area  of  Collaboration  Efforts  were  highlighted: 

•  Lack  of  accepted  OA  tools  to  assess  NCO  within  the  social  and  cognitive  domains 

•  Lack  of  pre-NCO  baseline  performance  analysis 

•  Lack  of  analysis  on  key  features  of  NCO  (important  to  the  international  community) 

•  NCO  is  stove-piped  (e.g.  by  service,  by  country,  by  discipline) 

•  International  sharing  of  data 

•  OA  analysis  timelines  are  not  fast  enough  to  meet  the  pace  of  technology  and  DOTMLPF 
development 

•  Developing  the  correct  measures  of  merit 

•  Cannot  adequately  represent  all  the  benefits  of  NCO  in  combat  models  (e.g.  self¬ 
synchronization  &  op-tempo) 

•  OA  is  overly  focused  on  military  to  the  exclusion  of  civilian  agencies 

•  OA  must  be  careful  to  represent  the  balance,  to  include  imintended  consequences  and 
characterizing  risks  and  benefits 
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Need  to  choose  OA  events  with  mutual  benefits  within  the  international  community 


Finally  in  the  area  of  applying  OA  to  actual  NCO  events,  the  shortfalls  discussed  were: 
warfighting  systems  are  not  designed  to  capture  OA  data,  the  instrumentation  to  capture  data  had 
to  be  developed  and  implemented  “on  the  fly,”  warfighting  systems  are  often  non-interoperable  - 
so  data  being  collected  would  not  be  in  any  kind  of  standard  format,  and  classification  and 
releasability  problems. 

SYNTHESIS  GROUP  FINDINGS 

After  participation  within  the  working  groups  and  detailed  discussions  in  their  meetings,  the 
Synthesis  Group  submitted  their  findings. 

WG 1 :  Measures  of  Merit  (MOMs)  -  The  findings  focused  on  the  assessment  that  the  MOMs 
for  NCO,  especially  for  understanding  a  networked  “cause  and  effect”  relationship,  are  a  work- 
in-progress.  However,  previous  MOMs  for  the  physical  domains  for  warfighting  missions  are 
well  understood  and  remain  relevant. 

•.  MOMs  must  both  enable  the  analyst  and  inform  the  client;  these  may  be  starkly  different  for 
Network  Centric  Operations  Analysis  (NCOA)  than  for  traditional  OA.  MOMs  for  NCO  A 
ai-e  a  work-in-progress. 

•  Survivability-related  MOMs  are  increasingly  relevant  to  military  operations  and  remain  so 
for  NCOA  as  well  (at  all  levels,  i.e.,  MOPs,  MOEs). 

•  A  combination  of  MOMs  is  necessary  for  OA  of  networked  “cause-and-effect”  relationships. 

WG  2:  Processes  and  Methodologies  -  The  findings  focused  on  the  NCW  conceptual 
framework  and  the  gap  between  sense-making  and  decision  making.  It  also  was  suggested  that 
the  relationship  between  cognitive  modeling  and  sense-making  modeling  needs  to  be  addressed. 

•  The  NCO  framework  emphasizes  “Quality  of  Individual  Sense-making”  and  the  “Quality  of 
Group  Sense-making”  versus  decision  making. 

•  Many  in  the  NCO  community  assert  that  cognitive  modeling  is  an  essential  part  of  the 
solution  to  modeling  sense-making. 

•  The  lack  of  processes  and  methodologies  for  OA  using  sense-making  and  cognitive  modeling 
make  the  NCO  firamework  a  challenge  to  operationalize. 

WG  3:  Assessment  Tools  -  The  findings  were  along  three  dimensions:  “better”  data, 
improvement  in  NCO  assessment  tools,  and  the  networking  of  assessment  tools  in  order  to 
achieve  synergy  in  the  application  of  a  tool  set. 

•  Some  tools  exist,  but  the  data  domain  (both  real  blue  and  red  data)  is  still  too  stove-piped. 

•  The  methods  to  change  information  to  knowledge  seem  to  exist  within  the  models,  but  are 
not  well  understood. 

•  Networking  assessment  tools  is  necessary  but  not  sufficient;  we  must  leam  to  “do  better 
things,”  not  “do  things  better.” 

•  The  networked  force  must  be  treated  as  a  holistic  entity  not  as  a  sum  of  discrete  elements. 

WG  4:  NCO  and  Force  Transformation  -  To  effect  Force  Transformation  it  was  concluded 
that  the  further  we  move  away  from  assessing  transformations  that  are  focused  on  a  single 
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Service,  tactics,  and  material  (“old”  missions,  etc.),  the  fewer  OA  tools  are  available  to  the 
analyst  for  information,  cognitive  and  social  domain  related  assessments. 

WG  5:  NCO  and  Coalition  Forces  -  It  was  discussed  that  the  OA  timelines  are  not  fast  enough 
to  keep  pace  with  technology  and  other  developments.  The  experience  of  the  UK’s  embedded 
OAs  needs  to  be  assessed  and  the  example,  maybe,  followed.  However,  the  inclusion  of  civilian 
agencies,  such  as  the  Department  of  Homeland  Security,  also  must  be  addressed. 

•  OA  analysis  timelines  are  not  fast  enough  to  meet  the  pace  of  technology  and  DOTMLPF 
development. 

•  OA  is  overly  focused  on  military  to  the  exclusion  of  civilian  agencies. 

•  Need  to  consider  new  level  of  embedded  analysis  (OA  in-theater  with  data  mining  and 
assessment  tools)  to  provide  real  time  feedback  on  NCO  effectiveness. 

WG  6:  Applying  OA  to  Actual  NCO  Events  -  It  was  found,  because  policies  and  doctrine  to 
incorporate  analysis  is  lacking,  along  with  the  minimal  use  of  embedded  analysts,  that  OA 
becomes  an  afterthought.  If  these  findings  and  the  data  domain  are  improved,  then  the 
assessment  of  actual  NCO  events  will  be  improved. 

•  Lack  of  joint  and  some  Service  doctrine  to  incorporate  analysis  in  a  JTF  limits  OA  support  of 
NCO. 

•  Reliance  on  non-embedded  analysts  makes  OA  an  afterthought  in  JTFs. 

•  OA  currently  limited  by  available  &  valid  data 

•  Time  and  data  constraints  limit  the  tool  set 

FUTURE  CHALLENGES 

The  working  group  participants  were  asked  to  assess  their  future  expected  challenges. 

The  primary  challenge  for  measures  of  merit  is  to  keep  up  with  the  rapidly  changing  scope  of 
NCO.  Agility  is  a  desirable  feature  (for  which  measures  are  needed)  in  NCO.  The  metrics  set 
should  be  agile  as  well.  Not  only  do  we  need  more  and  better  measures  for  NCO,  we  need  to 
rigorously  address  the  issue  of  sufficiency.  How  much  is  enough?  The  answer  may  differ  from 
one  situation  to  the  next.  Development  of  standards  for  NCO  analysis,  including  metrics,  faces 
the  challenge  of  incorporating  the  terminology  and  practices  of  the  social  sciences  in  order  to 
establish  measures  in  the  social  and  cognitive  domains. 

In  order  to  represent  NCO  accurately,  a  new  set  of  models  or  a  new  way  of  interoperating 
between  them  needs  to  be  created  and  all  aspects  of  NCO  need  to  be  represented.  This  means 
networking  not  only  two  battalions  but  also  all  actors  involved  in  the  operation  including  reach- 
back  capabilities.  This  is  the  essence  of  NCO,  and  the  capabilities  cannot  be  explored  if  they  are 
not  represented.  Essentially,  the  tools  that  need  to  be  developed  need  to  be  able  to  adapt  to 
changing  and  emerging  concepts.  In  a  complex,  network  centric  world  the  analysis  challenges 
will  continue  to  evolve.  We  should  not  adopt  a  mindset  of  getting  the  tools  and  processes  “right” 
-  we  need  an  ethos  of  continuously  examining  our  tools  and  processes  to  make  them  better. 

What  is  the  next  transformational  capability  or  technology  on  the  horizon?  Analysts  can  and 
should  be  engaged  in  this  effort,  and  clearly  must  look  beyond  the  bounds  of  defense  industries 


21 


for  promising  opportunities.  By  taking  a  broad  look  at  the  eight  “key  elements”  of 
transformation  (and  beyond)  we  may  uncover  areas  for  further  investigation.  This  is  not  a  one- 
player  game  -  we  also  must  think  about  areas  that  potential  adversaries  can  exploit  and  work 
them  in  to  our  future  scenarios  and  operating  concepts.  Using  Palm  Pilots™  on  the  battlefield  is 
an  example  of  bottom-up  approaches  that  have  been  attempted  by  tactical  and  operational  forces. 
These  may  not  be  officially  sanctioned.  Some  may  have  great  value,  some  may  have  minimal 
impact,  and  some  may  be  downright  dangerous.  Analysts  need  to  engage  in  the  effort  to  assess 
these  “bottom-up”  initiatives. 

Coalition  warfare  analysis  is  lacking  in  today's  assessment  tools,  hi  the  United  States,  “Joint”  is 
making  some  headway,  but  most  operations  in  the  future  will  include  Coalition  forces. 
Throughout  recent  history,  the  ability  to  share  information  in  a  Coalition  environment  has  been 
difficult  though  necessary.  Other  issues  that  need  to  be  addressed  in  the  Coalition  environment 
are  language  and  systems  interoperability.  NCO  most  certainly  is  about  fostering  collaboration 
and  networking  to  a  wide  degree.  We  must  expand  our  network  of  analysts  to  include 
interagency,  Coalition,  and  non-governmental  partners. 

On  the  battlefield,  if  we  don’t  do  anything,  the  information-sharing  problem  will  actually  get 
worse  because  the  planning,  execution  and  analysis  processes  are  conducted  increasingly  on  the 
SIPRNET.  This  can  have  the  effect  of  excluding  our  Coalition  partners.  Over-classification  of 
planning,  deployment,  execution  and  sustainment  products  can  exacerbate  the  information- 
sharing  problem.  This  adds  to  the  declassification  work  that  the  staff  Foreign  Disclosure 
Officers  must  do  prior  to  transferring  products  to  coalition  info  sharing  systems  like  CENTRDC 
(Combined  Enterprise  Regional  Information  Exchange  System). 

The  warfighting  culture  needs  to  be  changed  to  accept  more  analysis.  This  will  require  changes 
in  doctrine  and  training  and  experience  of  leaders.  As  analysts  do  more  collection  and  analysis 
in  the  field,  the  procedures  for  data  collection  will  mature,  but  emphasis  on  collecting  data 
should  start  in  the  planning  phase. 

EXTENT  OF  COLLABORATION 

As  with  the  State  of  the  Practice,  each  of  the  working  groups  were  asked  to  provide  a  self- 
assessment  of  the  level  of  collaboration  among  the  Services,  Coalition  Forces  and  US  Agencies 
where  operations  analysis  has  been  applied  to  NCO. 

The  level  of  collaboration  among  (and  within)  the  Services  was  assessed  as  inconsistent. 
Collaboration  between  services  is  varied  and  has  two  dimensions:  between  operators  and 
between  analysts.  Collaboration  between  analysts  was  good  with  the  Joint  Lessons  Learned 
Team  because  they  were  located  at  all  Combatant  and  Component  Headquarters  -  so  contact  was 
easy.  Collaboration  between  Service  teams  at  disparate  sites  was  very  infrequent.  Most  were 
completely  unaware  of  the  others’  existence. 

The  collaboration  with  the  US  Agencies  was  difficult  to  assess  because  the  majority  of  the 
attendees  did  not  know  of  examples.  Working  Group  6  discussed  that  the  GIF  Phase  IV 
planning  involved  almost  every  major  agency  in  the  US  government;  however,  none  of  these 
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agencies  had  the  collaborative  capabilities  and  expertise  to  contribute  effectively  and  in  a  timely 
manner. 

For  the  Coalition  Forces,  participants  considered  collaboration  to  be  infrequent  and  inconsistent. 
A  spectrum  of  change  of  military  forces  in  the  international  community  exists  from  status  quo  to 
modernization  through  transformation  of  forces.  Nations  are  at  different  points  in  the  spectrum 
with  respect  to  NCO,  but  all  are  committed  to  moving  away  from  the  status  quo  end  of  the 
spectrum.  The  participation  in  various  multinational  forums  is  evidence  of  the  commitment.  It 
was  noted  that  most  collaboration  with  Coalition  forces  is  on  a  bilateral  basis.  Many  of  the 
activities  are  modest  events  between  coalition  partners  (e.g.,  entropy  work,  agent  based 
modeling,  S&T  assessment,  an  assessment  of  decision  making  in  collaborative  environments, 
development  of  a  conceptual  model,  and  working  groups  to  discuss  interoperability).  The 
conclusion  is  that  there  still  is  a  need  for  significant  collaborative  analytical  events  between 
Coalition  partners. 

COALITION  COUNTRIES 

The  NCO  and  Coalition  Force  Working  Group  assessed  the  state  of  the  practice,  the  “good  and 
bad”  of  applying  OA  to  support  NCO,  and  the  future  challenges  for  four  Coalition  partners:  UK, 
Canada,  Sweden,  and  Australia. 

UNITED  KINGDOM  -  The  UK  believes  that  a  networked  force  will  allow  improved 
situational  awareness,  agility  and  synchronisation.  Although  a  range  of  Concepts  has  been 
developed  for  NEC  (covering  areas  such  as  the  development  of  a  Joint  Operational  Picture, 
collaborative  planning,  mission  organised  force  elements,  effects  based  planning  and  enhanced 
tempo)  the  main  focus  of  NEC  investment  is  currently  in  the  arena  of  experimentation. 

The  OA  community  has  a  wide  array  of  tools  available  to  undertake  analysis  in  support  of  NEC 
activities.  These  cover  the  spectrum  from  discrete  event  simulation,  classical  OA  tools,  sampling 
and  surveys  through  to  multi  dimensional  criteria  analysis,  picture  models  and  soft  systems 
methodologies.  The  UK  has  confidence  in  the  tools  for  assessing  issues  such  as  picture  quality 
and  timeline  comparisons  at  the  tactical  level,  and  is  moving  towards  analysis  of  some  of  the 
cognitive  issues  associated  with  situational  awareness.  There  are,  however,  still  significant  gaps 
in  the  assessment  of  the  quality  of  information  products  at  the  higher  level.  There  are  also 
limitations  in  aggregating  information  and  knowledge  factors  between  low-level  models  and 
high-level  models  and  hence  difficulties  in  the  calibration  of  tools  at  the  Campaign  level.  Current 
analysis  often  fails  to  represent  the  baseline  case  (for  example  not  accounting  for  the  many 
stove-piped  stand-alone  systems  that  currently  contain  the  data  used  by  decision  makers)  and  the 
resultant  limitations  in  showing  the  tme  worth  of  integrated  solutions. 

The  future  challenges  are  to  increase  the  availability  of  tools  and  metrics  for  addressing  issues 
such  as  quantification  of  information  quality  and  to  develop  tools  within  the  cognitive  domain  to 
handle  adaptive  behaviour  and/or  emergent  properties.  A  key  problem,  which  must  be  addressed, 
is  how  to  deal  with  Balance  of  Investment  issues  above  the  immediate  NEC  level,  where  nations 
have  to  trade  networked  enabled  capabilities  at  the  expense  of  platforms  or  other  hardware. 
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CANADA  -  While  there  have  been  some  NCO-related  initiatives  within  Canada,  such  as  the 
development  of  an  Integrated  ISR  Architecture,  currently  there  is  no  national  decision  regarding 
the  adoption  of  NCO.  A  high-level  departmental  symposium  scheduled  for  Fall  2004  is  expected 
to  resolve  this  and,  hopefully,  result  in  the  development  of  a  roadmap  for  the  way  ahead. 
Meanwhile,  a  series  of  UAV  experiments  at  the  Canadian  Forces  Experimentation  Centre,  using 
some  OA  tools,  are  working  towards  proving  the  NCO  case. 

The  use  of  planned  experiments  in  examining  the  merits  of  NCO  has  proven  useful.  For 
instance,  difficulties  with  communications,  TTP’s,  and  UAV  icing,  which  may  not  have  been 
discovered  otherwise,  were  discovered  through  experimentation.  However,  the  pace  of 
experimentation  has  provided  challenges  for  the  completion  of  OA  before  resources  are 
committed  to  the  next  in  the  series  of  UAV  experiments. 

Future  challenges  remain  for  Canada  in  the  area  of  NCO.  For  instance,  in  view  of  Canada’s 
participation  in  international  coalitions,  questions  remain  about  the  impact  of  such  considerations 
as  language,  culture,  policy  differences,  technology  and  TTP.  Also,  the  role  of  human  factors 
and  the  cognitive  domain  for  NCO  need  to  be  explored.  Importantly,  especially  in  view  of 
Canada’s  participation  in  coalition  warfare,  the  issue  of  how  trust  is  transmitted  across  an  NCO 
environment  between  coalition  partners  needs  to  be  explored.  Moreover,  questions  exist  about 
the  use  of  NCO  across  the  full  spectrum  of  violence  (i.e.  during  low  intensity  conflict,  peace 
support  operations,  etc.). 

SWEDEN  -  The  current  main  NCO-related  OA  effort  at  the  Swedish  Defence  Research  Agency 
(FOI)  is  the  support  to  the  Swedish  Network-based  Defence  experimentation  program,  where 
FOI  is  tasked  to  evaluate  the  ongoing  system  demonstrator  program.  FOI  also  supports  the  long¬ 
term  planning  process  within  the  Swedish  Armed  Forces  Headquarters,  providing  analysis  in 
support  of  the  Long-term  Parliamentary  Defence  Decision  2004.  FOI  was  tasked  with 
developing  and  providing  coarse  cost  estimates  of  alternative  network  structures,  coupled  to 
different  future  force  structures.  The  main  emphasis  has  been  on  the  network  components 
necessary  to  ensure  a  joint  networking  capability.  Early  in  that  work  we  pointed  to  the  basic 
balance  of  investment  between  networking  soldiers,  platforms  and  C2  systems,  respectively. 

Given  limited  resources  and  a  general  lack  of  large,  suitable  simulation  models,  they  are  forced 
to  focus  more  on  soft  OA  methods,  literature  studies  and  simple  models.  They  take  a  broad  view 
of  the  role  of  networks  in  the  field  of  Defence  Analysis,  which  can  be  outlined  in  the  following 
four  aspects;  Networks  as  enablers.  Networks  as  threats.  Networks  as  targets.  Networks  as 
tools.  The  first,  networks  as  enablers,  is  of  course  the  main  aspect,  and  encapsulates  all  their 
current  efforts  to  leverage  the  power  of  networks  in  developing  new  military  capabilities  and 
concepts  of  operation.  The  second  aspect  covers  the  idea  that  various  criminal  and  terrorist 
groups  are  effectively  early  adopters  of  networking.  If  you  believe  this,  a  better  understanding  of 
network  forms  of  organization  and  related  concepts  of  operation  is  required  not  only  to  transform 
our  armed  forces,  but  also  to  understand  the  character  of  future  conflicts  in  general.  The  third 
aspect  raises  the  obvious  fact  that  in  network-centric  operations,  your  networks  will  be  prime 
targets  for  enemy  attack.  More  broadly,  it  recognizes  that  all  our  values  at  risk  in  future  conflicts 
are  increasingly  networks  per  se\  civilian  infrastructure  is  largely  an  interconnected  set  of  co- 
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dependent  networks.  The  proper  way  to  identify,  evaluate  and  then  mitigate  these  new 
vulnerabilities  is,  however,  far  from  obvious. 

The  fourth  aspect  ties  it  all  together  and  starts  with  the  simple  question:  Do  they  as  OR  analysts 
have  the  proper  tools  and  the  necessary  understanding  to  deal  with  all  other  aspects  of  network 
centric  operations  in  the  future?  Project  Metanet  is  a  multi-year  competence-building  activity  to 
incorporate  recent  findings  from  such  diverse  areas  as  complexity  studies,  bioinformatics,  and 
Social  Network  Analysis,  into  work  at  the  Swedish  Defence  Research  Agency. 

AUSTRALIA  -  Network  Centric  Warfare  (NCW)  is  recognised  as  a  key  element  in  maximizing 
the  Australian  Defence  Force  (ADF)  military  effectiveness  both  the  Chief  of  Defence  Forces 
vision  for  the  ADF,  Force  2020  and  Future  Warfighting  Concepts,  documents.  The  latter 
document  describes  NCW  as  such:  “Network-centricity  will  help  us  to  link  national,  ADF  and 
coalition  sensors,  engagement  systems  and  decision  makers  into  an  effective  and  responsive 
whole.  At  its  core,  NCW  seeks  to  provide  the  future  force  with  the  ability  to  generate  tempo, 
precision  and  combat  power  through  shared  situational  awareness,  clear  procedure,  and  the 
information  connectivity  needed  to  synchronize  our  actions  to  meet  the  commander’s  intent. 

To  better  understand  and  assess  the  benefits  and  vulnerabilities  of  networked  operations,  and  to 
provide  timely  advice  to  our  senior  decision  makers  on  balance  of  investment  (e.g.  NCW  versus 
weapons),  we  need  to  carry  out  operations  analysis  (OA)  using  a  range  of  tools.  In  Australia,  we 
are  defining  the  concept  of  NCW  via  concept  development  and  experimentation.  We  have  found 
that  small  scale  experiments  in  the  form  of  seminar  war  games  and  matrix  games  to  be 
particularly  useful  for  gaining  insights  into  NCW.  We  also  are  developing  agent-based 
techniques  for  modeling  NCW.  The  value  of  quantifiable  metrics  for  knowledge  assessment  is 
also  recognized.  Our  current  advantage  is  in  our  comparatively  good  understanding  of 
information  exchange,  skills  in  conducting  small-scale  experiments  and  using  systems  of  systems 
approach  to  analyze  NCW.  On  the  other  hand,  we  note  that  many  more  tools  have  yet  to  be 
developed  to  assess  the  benefits  of  NCW. 

Some  of  the  challenges  include  the  inclusion  of  social  network  analysis  techniques  in  our  NCW 
analysis,  the  need  to  broaden  OA  capability  to  multi-disciplinary  areas,  including  social  and 
cognitive  modeling.  A  proper  representation  of  knowledge,  e.g.  via  fusion  of  data  into 
knowledge,  remains  elusive. 

RECOMMENDATIONS 

First,  the  Synthesis  Group  developed  a  set  of  actionable  recommendations  from  the  working 
group  discussions  and  their  findings. 

•  A  MORS  Symposium  working  group  or  composite  group  dedicated  to  NCO  is  needed.  NCO 
is  much  more  than  C4ISR  and  Information  Operations.  This  new  working  group  or 
composite  group  could  be  charged  to  examine  the  OA  toolkit  and  processes  in  order  to  better 
focus  on  the  NCO  problem.  At  the  72"*^  MORSS,  it  is  proposed  that  the  CG  B  Chair  discuss 
this  issue  with  the  leadership  from  Working  Groups  5-10,  providing  recommendations  to  the 
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MORS  Working  Group/Composite  Group  Committee  Chair  and  the  Vice  President  (Meeting 
Operations). 

•  Create  a  center(s)  of  excellence  within  services  and  JFCOM  for  NCO  assessments  to  capture 
and  share  results  and  data  of  experiments,  training,  analysis  and  experience. 

•  Enable  (e.g.,  thru  agreements,  business  practices)  the  networking  of  analysts  working  NCO 
and  provide  reach-back  for  tools,  data,  and  previous  studies. 

•  Establish  a  “business  model”  that  enables  services  to  conduct  operations  analysis  in  support 
of  “bom  joint”  NCO  concepts  and  that  is  underpinned  by  a  set  of  approved  tools  and  data. 

•  Conduct  follow-on  MORS  special  meetings  to  share  NCO  analysis  and  experimentation 
results.  There  were  two  options  recommended: 

-  A  2-day  or  3-day  Mini-Symposixom  to  educate  the  community  in  the  area  of  NCO 
analyses.  Training  and  education  was  seen  to  be  critically  important  to  Transformation 
overall,  and  specifically,  to  the  operations  analysis  practice  in  support  of  NCO. 

-  A  second  3-day  Workshop  to  discuss  the  issues  more  fully.  It  was  suggested  to  provide 
foundation  presentations  during  the  morning  of  the  1®‘  day  and  outbriefs  during  the 
afternoon  of  the  3'"*^  day  -  this  would  leave  two  full  days  for  discussion  and  further 
debate.  During  this  initial  workshop,  the  working  groups  felt  they  only  “scratched”  the 
surface  in  their  discussions. 

Each  of  the  working  groups  individually  suggested  some  additional  recommendations. 

For  the  Measures  of  Merit  Working  Group,  one  recommendation  was  to  take  a  functional  area, 
such  as  fires,  and  add  a  qualitative  dimension  beyond  existing  metrics  in  the  physical  domain. 
Next  develop  metrics  related  to  fires  that  address  content  for  the  information  domain  and  then 
learn  how  to  deal  with  measuring  functions  related  to  fires  in  the  cognitive  and  social  domains. 
Another  recommendation  was  to  consider  using  “negative”  measures  of  merit  to  examine 
phenomena  that  indicate  problems  are  developing. 

For  Processes,  Methodologies  and  Assessment  tools,  the  first  recommendation  (and  challenge) 
was  offered  by  VADM  Cebrowski  during  his  keynote  address  -  develop  tools  for  NCO.  Other 
recommendations  included  to:  invest  in  tool  development  for  assessing  C4ISR  systems,  continue 
research  on  sense-making  and  cognitive  modeling,  consider  revising  the  NCO  fi-amework  to 
focus  on  “Quality  of  Individual  Decision  making”  (to  enable  decision  superiority),  allow  sense¬ 
making  and  cognitive  modeling  to  compete  with  other  modeling  techniques  in  the 
implementation  of  NCO  modeling,  and  identify  expertise  in  social  network  analysis  tools  and 
appropriate  representation  of  cognitive  processes  to  integrate  them  expertise  into  current  and 
future  tools.  Additionally,  the  issues  of  accessibility,  releasability,  and  classification  must  be 
resolved. 

In  the  area  of  Force  Transformation,  it  was  recommended  to  expand  the  analysis  community 
beyond  the  traditional  competencies  and  include  disciplines  with  expertise  in  the  eight  key 
elements  of  the  force  (concepts,  leadership,  education,  culture,  technology,  organization,  process, 
training).  Also,  suggest  continuing  to  utilize  cross-disciplinary  analysis  teams  to  address 
complex  NCO  issues.  As  we  begin  to  make  headway  on  NCO  analysis  techniques  we  must 


share  new  problem  solving  approaches  and  bring  additional  disciplines  into  the  OA 
“mainstream.” 

For  the  Coalition  Forces,  the  recommendations  were  derived  primarily  from  the  ‘future 
challenges’  identified  by  each  individual  country  and  agreed  to  by  the  entire  group  as  applicable 
to  all  coalition  countries.  These  recommendations  included: 

•  Ensure  lessons  learned  teams  have  OA  team  members. 

•  Develop  processes  to  collect  data  from  real  world  operations,  experiments  and  training,  and 
provide  feed-back  into  NCO  analysis. 

•  In  the  social  and  cognitive  domain  issues  of  NCO,  develop  OA  methods  and  metrics  for 
assessment  and  create  opportunities  for  experiments 

•  Assess  biological  and  other  adaptive  systems  network  analysis  tools  for  applicability  to 
NCO. 

•  Advocate  the  use  of  case  study  methodology  in  the  analysis  of  NCO;  improve  the  rigor  of 
analysis  of  case  studies  and  experiments  (hypothesis  testing,  analysis  plans,  collection  and 
dissemination). 

•  Develop  quantitative  techniques  for  assessing  the  quality  of  information. 

•  Conduct  an  assessment  of  interoperability  requirements  in  NCO  (joint.  Coalition,  and 
interagency). 

•  Conduct  an  increased  emphasis  on  NCO  model  validation. 

•  Conduct  analysis  on  attack  and  defense  of  networks. 

When  applying  OA  to  actual  NCO  events,  recommendations  focused  on  developing  doctrine, 
plans  and  processes  to  network  OAs  and  include  them  early  in  the  planning.  These  included 
developing  products  and  processes  to  facilitate  NCO  information  sharing  in  order  to  network 
analysts  with  collaborative  capabilities,  developing  methodologies/tools  to  enhance  OA  support 
within  time  constraints  of  the  actual  event,  networking  analysts  so  that  they  can  exchange 
techniques  and  data,  and  coding  billets  in  the  SJFHQ  for  OA  analysts. 

Finally,  the  participants  made  other  general  suggestions: 

•  Develop  a  code  of  best  practices  for  NCO. 

•  Encourage  continued  training  and  education  related  to  NCO  concepts,  and  supporting 
analyses  (models,  simulations,  experiments) 

-  Analysts  should  examine  what  knowledge  and  processes  can  be  drawn  from  non- 
traditional  disciplines 

-  Training  and  education  was  seen  to  be  critically  important  to  transformation  overall,  and 
specifically  to  the  OA  practice  in  support  of  NCO.  This  was  the  same  for  government, 
military  and  industry;  and  was  felt  to  be  the  same  internationally  as  well  as  in  US 

-  OA  needs  to  get  back  to  its  multi-disciplinary  roots.  Need  to  seek  out  social  sciences  as 
well  as  the  hard  sciences  and  engineering  fields  for  ways  to  abstract  and  approach 
squishy,  and  seemingly  intractable  problems. 

•  Expand  tool  kit  and  develop  more  flexible  processes  to  examine  robustly  networked 
problems  and  complex  adaptive  systems. 
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•  Refine  existing  tools  (better,  faster,  cheaper)  to  meet  compressed  decision  cycles. 

•  Reinvigorate  the  use  of  war  gaming  for  analysis  (only  tool  that  can  represent  the  complex 
adaptive  system  -  need  a  thinking,  adaptive  adversary  that  is  allowed  to  win) 

•  Rely  less  on  formal  events  and  more  on  real  world  data  collection  and  exploitation. 

•  Address  the  fundamental  questions:  (1)  What  is  the  war  fighting  value  of  information? 

(2)  Is  there  a  candidate  list  of  metrics  for  conduction  NCO  analysis?  (3)  What  are  the 
Wdamental  characteristics  of  the  NCO  problem? 

•  For  the  OA  community  to  improve  the  state  of  practice  in  NCO,  some  additional  steps  are 
recommended. 

-  It  is  recommended  that  the  resources  needed  to  develop  and  sustain  improvements  should 
come  through  case-studies  and  experimentation  initiatives  in  diverse  operational  contexts 
and  scales. 

•  Given  the  identified  need  to  adapt  paradigms,  techniques,  etc.,  from  the  social  and  biological 
sciences,  recommend  the  services  pursue  complexity  science,  complex  adaptive  systems 
applications  to  NCO  -  OA  problems  at  the  tactical  and  operational  levels,  including 
developing  or  adapting  metrics  and  tools  from  non-military  /  social  science  contexts.  Further 
recommend  MORS  consider  the  opportunity  of  'networking'  professionally  across  other 
communities  of  practice  to  formalize  exchanges  and  build  more  interdisciplinary  tools  and 
practices. 

•  Visible  and  more  enduring  forums,  beyond  a  single  workshop,  might  assist  in  advancing  the 
OA  community's  efforts  with  respect  to  education  /  training  —  what  some  refer  to  as 
"knowledge  sharing".  In  the  spirit  of  improving  awareness,  increasing  documentation,  and 
broadening  relationships,  perhaps  a  NCO-OA  "Community  of  Practice  or  Interest"  could  be 
chartered.  This  would  provide  a  means  for  bringing  together  folks  from  various  communities 

-  CCRP,  MORS,  and  Coalition  countries,  as  well  as  multidisciplinary  scientific  groups  —  so 
they  could  COLLABORATIVELY  (and  virtually)  engage  and  "network"  around  the  various 
issues  via  questions/discussion  groups,  key  links,  papers  and  studies  /  POCs,  aligning  across 
various  dimensions  of  the  problem  (e.g.,  key  themes  /  needs  identified  in  the  workshop 
dialogue  /  construct). 

•  Recommend  creating  disincentives  and  incentives  aimed  at  'changing  the  behavior'  of  the  OA 
community  to  better  respond  to  NCO  related  analysis  needs.  Developing  and  advocating  a 
Code  of  Best  Practice  for  NCO  may  address  this,  but  only  to  a  limited  extent.  To  accelerate 
the  state  of  practice  in  NCO  OA  will  still  require  overcoming  rather  steep  "cultural"  barriers 
where  mindsets  (and  investments)  are  wedded  to  "traditional"  practices  and  tools. 

Recognition  of  "great  work"  in  NCO  OA  may  be  something  the  MORS  Awards  Committee 
could  consider. 

OTHER  CONSIDERATIONS 

In  the  area  of  applying  operations  analysis  to  Network  Centric  Operations,  the  participants 
offered  a  couple  of  other  considerations. 

It  is  easy  to  get  the  impression  that  “faster  is  better”  always  applies  in  NCO.  A  higher 
OPTEMPO  is  one  of  the  advertised  benefits.  Speed  is  certainly  better  in  the  information  domain, 
and  in  the  physical  domain  once  an  action  has  begun.  However,  this  may  not  always  apply  in  the 
cognitive  and  social  domains.  A  higher  OPTEMPO  may  not  be  the  answer  in  all  cases.  In  a 
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broader  context,  the  issue  to  be  addressed  might  be  “Who  is  deciding  when  and  where  to  engage 
and  who  is  in  a  reactive  mode?” 

Analysts  need  to  portray  their  subjects  of  analysis  in  terms  of  applicable  architectures.  This  will 
facilitate  assessing  the  degree  of  their  impact  on  capabilities.  The  Operations  Analysis  Process 
should  be  updated  and  expanded  to  reflect  where/how  architecture  analysis  and  the  more 
classical  pyramid  of  Modeling  and  Simulation  fit  together  in  an  evolutionary  concept 
development  or  exploratory  analysis  of  NCO. 

Finally,  it  was  proposed  to  document  lessons  learned  from  initial  NOW  studies. 

SYNTHESIS  GROUP  SUMMARY 

The  Synthesis  Group  summarized  their  findings  thus: 

The  good  news  is  that  the  OA  process  developed  in  the  Industrial  Age  has  continued 
applicability  to  Network  Centric  Operations  (NCO)  in  the  Information  Age.  We  are  only 
beginning  to  understand  the  practices  that  should  be  followed  to  perform  credible  NCO  analyses. 
In  addition,  we  know  there  are  NCO  considerations  that  remain  challenges.  The  bad  news  is  that 
we  do  not  do  a  satisfactory  job  in  documenting,  sharing,  and  learning  from  NCO  analyses 
performed. 

The  worse  news  is,  that  with  the  arrival  of  terrorism,  other  asymmetric  threats,  rogue  nations, 
and  increased  concern  for  homeland  defense,  NCO  provides  an  opportunity  for  Force 
Transformation.  Although  these  are  daunting  challenges,  there  is  some  better  news  that  is 
implicit  in  this  workshop  itself.  If  the  recommendations  of  this  workshop  are  implemented 
successfully,  they  will  provide  a  basis  for  improving  the  operations  analysis  process  and 
empowering  the  OA  community  to  address  the  most  critical  analytical  challenges  to  support 
NCO  and  Force  Transformation. 

SPONSOR  BRIEF  COMMENTS 

The  MORS  Sponsors  were  briefed  on  the  results  of  the  Workshop  and  provided  some  additional 
comments: 

•  The  Synthesis  Group  recommended  that  centers  of  excellence  be  created  within  services  and 
JFCOM  for  NCO  assessments  to  capture  and  share  results  and  data  of  experiments,  training, 
analysis  and  experience.  The  Sponsors  believed  that  separate  centers  for  NCO  should  not  be 
created,  but  NCO  should  be  the  principle  that  ties  other  centers  together. 

•  The  Synthesis  Group  provided  an  overall  qualitative  assessment  of  the  current  “state  of  the 
health”  of  applying  the  Operations  Analysis  Process  across  the  four  key  NCW  domains.  The 
Sponsors  further  discussed  the  cognitive  and  social  domains:  1)  each  commander  reacts  in 
different  ways;  2)  these  domains  focus  on  behaviors;  3)  excursions  should  be  analyzed  for 
different  leadership  styles;  and,  4)  these  domains  should  be  included  in  designing 
experiments. 
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WORKING  AND  SYNTHESIS  GROUP  REPORTS 


Working  Group  1  — Measures  of  Merit 

Working  Group  2  —  Processes  and  Methodologies 

Working  Group  3  —  Assessment  Tools 

Working  Group  4  —  NCO  and  Force  Transformation 

Working  Group  5  —  NCO  and  Coalition  Forces 

Working  Group  6  —  Applying  NCO  to  Actual  Events 

Synthesis  Group 
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Hilitatv  Operations  Research  Society 

Operations  Analysis  Support  to 
Network  Centric  Operations 


Military  Operations  Research 
Society  (MORS)  Workshop 

WG  #1 :  Measures  of  Merit 
Outbrief 
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WG 1  -  Measures  of  Merit 


m€)H3 

HifftaiY  Operations  Research  Society 


•State  of  the  Practice 

♦  Level  -  Frequent  use,  inconsistent  application  (hybrid  of  Levels  4 
and  5) 

♦  Applications/Examples  - 

FCS  Milestone  B  AoA 
FCS  C2  Program 

Joint  Expeditionary  Force  Experimentation  (JEFX) 

Single  Integrated  Air  Picture  (SIAP)  System  Engineering  Analysis 
Mission  Planning  System  for  the  Common  Undersea  Picture 

♦  OA  Process  - 

Define  measures  for  evaluating  alternatives  that  are  consistent  with 
the  decision  maker’s  definition  of  the  problem.  The  metrics  set  in  each 
case  depends  on  the  problem  set. 

-  Operational  architecture  (acquisition)  decisions 

-  Operational  decisions  within  an  architecture 

-  Doctrine 


Working  Group  1  was  polled  to  establish  the  level  of  OA  use  in  support  of  NCO. 
The  result  was  a  hybrid  score  between  Level  4  (Inconsistent  Use)  and  Level  5 
(Frequent  Use)  -  call  it  a  4.5  (Frequent  Use,  Inconsistent  Application). 

Army,  Navy  and  Joint  programs  are  cited  as  examples  of  OA  applications  to 
NCO. 

The  selection  of  Measures  of  Merit  (MoMs)  falls  into  the  analysis  area  in  the  OA 
process.  Measures  are  chosen  that  are  appropriate  to  the  question  being 
addressed. 
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WG 1  -  Measures  of  Merit 


mms 

Military  Operations  Researdi  Society 


•Key  Issues,  Shortfalls  and  Gaps 


♦  Key  Issues 

The  taxonomy  of  NCO  metrics  is  in  a  highiy  theoretical  state.  A  more 
detailed  structure  is  needed.  Examples  would  support  wider 
understanding  and  acceptance. 


Measures  associated  with  the  Lanchesterian  approach  are  not 
sufficient  to  capture  the  effects  of  NCO. 

-  Traditional  measures  of  attrition  are  not  appropriate  for  many  applications. 

-  Rates  are  more  important  than  totais  in  many  cases. 

-  Survivability  of  own  forces  is  increasing  in  value  as  a  measure. 

-  Qualitative  measures  are  already  in  use  at  the  operational  level. 

-  Quantitative  measures  of  quality  are  needed,  especially  in  the  information 
domain. 


Metrics  generally  focus  on  “measures  of  goodness.”  It  could  be  useful 
to  formulate  measures  for  negative  influences. 


A  taxonomy  of  metrics  is  needed  to  enhance  OA  support  to  NCO.  There  are 
several  ways  to  organize  NCO  measures  that  make  intuitive  sense  but  examples 
of  specific  measures  within  these  categories  are  hard  to  find. 

“Traditional”  measures  are  necessary  but  not  sufficient  to  address  the  needs  of 
analysts  in  the  NCO  area.  Measures  of  attrition,  such  as  LER,  do  not  apply  to  all 
situations  in  NCO  analysis.  The  number  of  enemy  kills  may  not  be  a  strong 
indicator  of  success.  A  qualitative  aspect,  such  as  that  offered  by  the  number  of 
High  Priority  Targets  (HPTs)  killed  may  be  more  servicable. 

Survivability  of  own  forces  has  increased  in  importance  as  an  NCO  measure. 

This  standard  can  be  applied  to  enemy  forces  as  well,  i.e.,  average  survival  time 
of  an  HPT. 

In  general,  qualitative  measures  should  be  accepted  for  some  purposes. 
Qualitative  assessments  (red,  orange,  blue,  green)  were  used  in  this  workshop. 

Metrics  generally  focus  on  degrees  of  achievement  of  desirable  properties  related 
to  systems.  Additional  metrics  that  address  unfavorable  developments,  which 
may  not  be  associated  with  specific  systems,  could  be  useful. 


33 


fVG  1  -  Measures  of  Merit 


m@>H3 

MUItary  Operations  Itesearch  Society 


•Key  Issues,  Shortfalls  and  Gaps 

♦  Key  Shortfalls/Gaps 

Currently  available  metrics  fall  mostly  into  the  physical  and 
information  domains.  There  are  gaps  in  the  cognitive  and 
social  domains. 

-  Physical  measures  necessary  but  not  sufficient,  may  be  more 
appropriate  as  MOPs. 

-  We  need  to  obtain  the  participation  of  professionals  who  can 
contribute  to  the  development  of  metrics  in  the  cognitive  and  social 
domains. 

^  There  are  more  bilateral  NCO  tools,  experiments  and  exercises 
than  multilateral. 


Most  metrics  in  current  use  for  NCO  fall  into  the  physical  and  information 
domains.  This  is  not  to  say  that  existing  measures  in  these  domains  fill  all  the 
needs  there,  but  there  is  more  choice  (and  understanding)  in  these  two  areas  than 
in  the  cognitive  and  social  domains.  The  participation  of  social  scientists  should 
be  sought  to  assist  in  addressing  this  need. 

Useful  measures  are  constantly  being  developed  for  analytical  efforts  but  they 
are  not  widely  disseminated  across  the  OA  community.  This  is  due  in  part  to  the 
fact  that  most  analysis  is  conducted  on  a  bilateral  basis,  at  best.  More 
multilateral  projects  are  needed  among  the  services  and  with  coalition  partners. 
Joint  staff  and  JFCOM  studies,  while  joint  in  nature,  do  not  necessarily  involve 
the  services  and  allies  sufficiently  to  achieve  cross  pollenization  across  the  OA 
community. 


34 


fVG  1  -  Measures  of  Merit 


Military  Operations  Research  Society 


•Future  Challenges 

♦  How  can  metrics  be  kept  in  sync  with  the  dynamic  nature  of  NCO 
concepts? 

♦  How  do  we  measure  NCO  element  sufficiency? 

♦  How  are  standards  to  be  developed? 

Languages  across  disciplines 
Interfaces 

♦  How  can  we  establish  NCO  within  the  analytic  community? 


The  primary  challenge  for  measures  of  merit  is  to  keep  up  with  the  rapidly 
changing  scope  of  NCO.  Agility  is  a  desirable  feature  (for  which  measures  are 
needed)  in  NCO.  The  metrics  set  should  be  agile  as  well. 

Not  only  do  we  need  more  and  better  measures  for  NCO,  we  need  to  rigorously 
address  the  issue  of  sufficiency.  How  much  is  enough?  The  answer  may  differ 
from  one  situation  to  the  next. 

Development  of  standards  for  NCO  analysis,  including  metrics,  faces  the 
challenge  of  incorporating  the  terminology  and  practices  of  the  social  sciences  in 
order  to  establish  measures  in  the  social  and  cognitive  domains. 

To  avoid  analytical  stovepiping,  which  could  occur  within  the  NCO  domains, 
standards  for  dealing  with  the  interfaces  among  the  domains  are  needed. 

We  need  to  find  a  way  to  establish  a  community  of  practice  for  NCO. 
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WG 1  -  Measures  of  Merit 

•Extent  of  Collaboration 

♦  Among  the  Services 

^  Level  -  4.8  Frequent  but  inconsistent 
Examples 

~  USMC  Project  Albert 
~  JFCOM 

-  JWARS 

-  MORS 

♦  With  the  Coalition  Forces 

Level  -  4.5  Less  frequent  and  also  inconsistent 
Examples 

-  NATO 

-  US/UK  Interoperability  Studies 

-  Other  bilateral  agreements 

♦  With  the  US  Agencies 

^  Level  “  2  Just  starting 
^  Example  -  JANUS  with  Border  Patrol  Reps 


MUitarY  OperatkMU  Research  Sodet/ 


Working  Group  1  was  polled  again  to  rate  the  extent  of  collaboration  in  NCO 
analysis.  Again,  we  felt  more  comfortable  with  hybrid  ratings. 

In  retrospect,  it  is  interesting  that  we  selected  JFCOM  as  an  example  of  inter¬ 
service  collaboration.  Individuals  from  all  the  services  are  assigned  to  JFCOM 
and  certainly  collaborate  within  the  command,  but  is  this  enough?.  While  these 
individuals  bring  a  service  oriented  perspective  to  their  JFCOM  duties,  they 
ultimately  speak  for  JFCOM,  not  their  parent  services. 

It  was  noted  that  most  collaboration  with  coalition  forces  is  on  a  bilateral  basis. 
We  had  a  difficult  time  finding  examples  of  interagency  collaboration. 
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WGl  Measures  of  Merit 

«/  MHtary  Operations  Research  Society 

Recommendations 

♦  Specific  to  WG  area 

^  Vertical  development  of  NCO  measures  is  sometimes  appropriate. 

-  Take  one  functional  area,  such  as  fires,  and  examine  existing  metrics  in  the 
physical  domain  for  completeness  with  respect  to  NCO. 

-  Develop  metrics  related  to  fires  that  address  content  for  the  information 
domain 

-  Learn  how  to  evaluate  functions  related  to  fires  in  the  cognitive  and  social 
domains. 

-  Integrate  laterally  with  other  functional  areas. 

Consider  using  “negative”  MOMs  to  examine  phenomena  that  indicate 
problems  are  developing. 

In  some  cases  where  traditional  metrics  apply  at  the  MOE  or  MOO 
level,  the  underlying  MOPs  used  to  capture  the  performance  of  a 
network  centric  force  may  be  different  from  those  used  to  characterize 
a  platform  centric  force. 

A  sample  taxonomy  for  NCO  measures  follows  for  consideration. 


Many  of  the  members  of  the  working  group  have  not  been  involved  in  NCO 
conceptual  development  at  the  higher  levels.  They  feel  more  comfortable 
applying  NCO  principles  to  a  restricted  scope  first,  then  integrating  with  other 
functional  areas. 

Instead  of  always  focusing  on  metrics  that  are  intended  to  measure  how  well  the 
force  is  doing,  it  could  be  useful  to  define  measures  that  provided  an  indication 
of  a  trend  toward  an  unfavorable  direction.  Such  measures  would  probably  not 
focus  on  how  well  the  force  is  doing  in  accomplishing  its  assigned  tasks,  but  on 
environmental,  cultural,  economic  or  other  indicators  of  developments  that  could 
make  the  job  more  difficult  for  the  force. 

There  may  be  a  common  measure  for  comparing  the  performance  of  a  network 
centric  force  with  that  of  a  platform  centric  force  at  the  MOE  level,  but  the 
MOPs  that  are  used  to  assess  each  of  the  two  forces  may  be  different.  This  may 
be  indicative  of  differing  concepts  of  operations. 
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Sample  Taxonomy  of  NCO  Measures  m@)H3 

_ _ _  Operations  Research  Society 


Measure  of  Success 

Activity /Pro  cess  | 

Policy/ 

Outcome 

Policy  Effectiveness 

Normalcy 

Process  9 

Force 

Effectiveness 

Mission  Effectiveness 

Degree  of  Agility 

Process  8 

Execution 

Sustainability  /  Tempo 

Extent  Commander's  Intent  Achieved 

Process  7 

Coordination 

Self-Synchronized  Forces 

Degree  of  Synergy  Among  Units 

Process  6 

Planning/Tasking 

C2 

Effectiveness 

Enhanced  Collaboration 

Degree  of  Commonality  of  Purpose 

Process  5 

interaction 

improved  Shared  Awareness 

Degree  of  Common  Perception 

Process  4 

Cognitive  Analysis 

Performance 

Improved  Quality  of  Info 

Degree  Relevant  Info  Available 

Process  3 

Augmentation/Redirection 

Shared  Information 

Degree  Relevant  Info  Available 

Process  2 

Distribution 

Robust  Network 

Connectivity 

Dimensional 

Parameters 

Process  1 

Network  Management 

Nodes/LInks 

Bandwidth 

Several  members  of  Working  Group  1  collaborated  to  produce  a  table  that  relates 
measures  and  processes  to  the  supposed  benefits  of  NCO.  Each  process  is  a  link 
between  a  pair  of  benefits.  The  benefits  are  arranged  in  ascending  order,  from 
least  to  greatest.  Concepts  discussed  hy  plenary  speakers  were  incorporated. 
Hopefully,  this  initial  attempt  will  stimulate  more  development. 
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WG 1  -  Measures  of  Merit 


HUIt»7  Operations  Research  SodeCy 


•  Recommendations 

♦  Specific  to  WG  area 

^  “Faster  is  better”  may  not  apply  across  the  board. 

-  It  probably  applies  more  generally  to  the  Information  domain  and  to 
the  physicai. 

»  Process  the  information  and  develop  plans  quickly,  then  execute  the 
most  advantageous  time. 

»  When  actions  are  undertaken,  they  should  be  performed  expeditiously. 

-  Perhaps  a  different  emphasis  is  needed  in  the  cognitive  domain. 

»  Instead  of  focusing  exclusively  on  higher  OPTEMPO,  consider  instead 
which  side  is  deciding  when  and  where  to  engage.  This  is  a  good 
example  of  a  qualitative  (as  opposed  to  quantitative)  measure 
that  could  be  useful  in  assessing  NCO. 


-  In  the  social  domain,  speed  may  be  counterproductive  if  team 
confidence  has  not  been  fully  established. 

♦  Other  Comments 


In  general,  qualitative  (subjective)  measures  of  merit  may  apply  more 
broadly  to  Effects  Based  Operations  (EBO)  than  to  NCO. 


It  is  easy  to  get  the  impression  that  “faster  is  better”  always  applies  in  NCO.  A 
higher  OPTEMPO  is  one  of  the  advertised  benefits.  Speed  is  certainly  better  in 
the  information  domain,  and  in  the  physical  domain  once  an  action  has  begun. 
However,  this  may  not  always  apply  in  the  cognitive  and  social  domains.  A 
higher  OPTEMPO  may  not  be  the  answer  in  all  cases.  In  a  broader  context,  the 
issue  to  be  addressed  might  be  “Who  is  deciding  when  and  where  to  engage  and 
who  is  in  a  reactive  mode?” 

This  is  a  good  example  of  a  qualitative  (as  opposed  to  quantitative)  measure  that 
could  be  useful  in  assessing  NCO.  However,  such  subjective  measures  of  merit 
may  apply  more  broadly  to  Effects  Based  Operations  than  to  NCO.  NCO  and 
EBO  are  linked  in  recent  writing  on  the  theory  of  information  age  warfare.  This 
interface  may  be  the  most  fertile  area  for  developing  qualitative  metrics.  Other 
aspects  of  NCO  cannot  adequately  be  characterized  without  quantitative 
measures. 
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WGl-  Measures  of  Merit 
•Recommendations 


MiUtafy  Operations  Research  Society 


♦  Overall  OA  Support  to  A/CO 

More  analytic  studies  are  needed  on  areas  of  common  interest  in  a 
joint/coalition,  collaborative  environment  (ex,  TTCP  Net  Centric 
Maritime  Warfare  Study) 

-  Netted  fires 

-  Netted  sensors 

-  Communications 

-  Logistics 

Develop  a  code  of  best  practices  for  NCO. 

^  More  opportunities  for  education  and  sharing  of  knowledge  on  NCO 
are  needed. 

-  Web  sites  with  bibliographies,  lessons  learned,  links 

-  Tutorials 

-  Collaboration  with  the  educational  community 

^  More”focused”  or  limited  experiments  are  needed  with  dedicated 
forces  not  involved  in  training.  This  should  aid  in  transformation,  as 
operators  innovate  with  new  tools  given  the  freedom  to  do  so. 


The  members  of  Working  Group  1  believe  addressing  these  issues  would 
enhance  NCO  analysis  across  the  board  and  allow  them  as  individuals  to 
participate  more  effectively. 
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NCW Four  Key  Domains 


nis>H3 

Kilitary  Operations  Research  Society 


-  Current  “State  of  the  HEALTH” 
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Current  “State  of  the  Health” 
for  OA  Process 


^ — r—: — 


isieasures  of  : 


Ij  ttuihan  Factors/^ 
i  C)rganizhti6h" 


WG 1  -  Measures  of  Merit 


niGiRS 

Military  Operations  (lesearch  Society 


•Briefings  Presented  During  WG1  Sessions 

♦  NCO  Conceptual  Framework  -  Dr.  Daniel  T.  Maxwell,  EBR 

♦  lERs,  the  Pk’s  of  Network  Modeling  -  Mr.  Peter  M.  Kerekanich, 
TRAC 

♦  WIN-T  Analysis  of  Alternatives,  Emerging  Results  -  Mr.  Steve 
Herndon,  TRAC 

♦  NC  Measures  of  Merit  from  a  Queueing  Theory  Perspective  -  Dr. 
Ralph  S.  Klingbeil,  NUWCDIVNPT 

♦  Quantifying  Information  Availability  for  Situational  Awareness  -  LTC 
Stephen  R.  Riese,  TRAC 


Briefs  presented  during  the  Working  Group  1  sessions  led  to  lively  discussions. 
Dr.  Dan  Maxwell  set  the  stage  by  describing  the  current  state  of  the  NCO 
concept.  Mr.  Pete  Kerekanich  then  gave  us  a  micro  level  (MOP)  look  at  the 
WIN-T  program,  examining  network  performance.  He  was  followed  by  Mr. 
Steve  Herndon,  who  shifted  the  focus  to  the  MOE  level,  illustrating  the 
difference  in  operational  outcomes  with  and  without  WIN-T.  Dr.  Ralph 
Klingbeil  gave  us  a  Navy  perspective.  It  was  interesting  to  note  that  his  queuing 
theory  approach  could  accommodate  both  changes  in  the  red  behavior  and  the 
changes  to  blue  CONOPS  without  the  need  to  find  new  MOPS.  LTC  Steve  Riese 
closed  the  presentations  with  a  stimulating  brief  on  the  effect  on  a  commander’s 
behavior  of  a  lack  of  information  about  red  deployment  (or  lack  of  confidence  in 
the  common  operating  picture). 

The  metrics  used  for  evaluation  of  NCO  capability  in  these  briefs  were  thought 
provoking. 


43 


44 


MKttary  Operations  Research  Society 


Operations  Analysis  Support  to 
Network  Centric  Operations 

Military  Operations  Research 
Society  (MORS)  Workshop 


WG  #2:  Operations  Analysis 
Methods  and  Process 


Outbrief 


45 


fVG  2  Objectives 


Military  Operations  Research  Society 


1 .  Use  the  NCO  Conceptual  Framework  to 
assess  operations  analysis  ability  to  evaluate 
NCO  enabling  capabiiities. 

2.  To  identify  areas  of  weakness  in  both  the 
NCO  Conceptual  Framework  and  the  OA 
Process 

3.  To  recommend  near  and  longer  term  actions 
that  wiil  improve  the  OA’s  community  to 
support  NCO  related  research,  analyses,  and 
implementation 


The  Overall  objective  of  Working  Group  2  was  to  focus  on  the  operations 
analysis  process  as  it  relates  to  network  centric  operations  and  related  operational 
concepts. 

The  NCO  Conceptual  Framework,  although  still  a  work  in  progress,  served  both 
as  a  guide  and  a  “point  of  departure”  for  discussions. 

This  approach  was  not  an  endorsement  of  the  fi-amework,  but  it  did  validate  its 
value  as  a  communications  vehicle  for  discussing  NCO  —  an  area  of  thought  that 
currently  lacks  formal  and  unambiguous  terms  of  reference. 

Additionally,  the  framework  highlighted  topic  areas  inside  of  operational 
analysis  that  fall  short  for  supporting  net-centric  operations,  developing  future 
network  centric  concepts,  and  efficiently  equipping  the  force  of  the  future. 
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Approach  HlOrtS 

MKItary  Operations  Research  Society 

•  Receive  framing  presentations 

♦  McIntyre,  JWARS 

♦  Maxwell,  NCO  Conceptual  Framework  Research  Program 

♦  Mathieson,  Full  Spectrum  Analysis 

♦  Worm,  Sweden’s  Roadmap  to  Network-Based  Defence 
Transformation 

•  Use  conceptual  framework  to  assess 
NCO  OA  capabilities 

♦  Decompose  and  assess  state  of  practice/capability  to  analyze 

•  Make  recommendations 


WG-2  was  composed  of  about  25  very  experienced  analysts,  both  military  and 
civilian. 

Essentially,  the  WG  attempted  to  make  an  assessment,  albeit  somewhat 
subjective,  of  the  state  of  knowledge  and  practice  in  the  OA  community  for 
dealing  with  the  challenges  of  NCO.  (NCO  was  recognized  to  span  the  entire 
spectrum  of  missions  for  which  military  forces  may  be  applied  to  in  the  future.) 
These  assessments  are  summarized  in  subsequent  charts. 

The  group  also  solicited  each  member  to  provide  one  (uncensored)  observation 
from  their  perspective  as  to  the  “state”  of  the  practice,  and  one  long-term  and  one 
short-term  recommendation  to  the  OR  community.  These  were  collated  into  a 
set  of  bullets  that  are  contained  in  the  backup  section  of  this  presentation. 

The  Backup  section  of  this  presentation  also  contains  a  set  of  “commented” 
slides  with  relevant  observations  about  the  strengths  and  weaknesses  of  the 
conceptual  framework,  and  OA  practice  relative  to  the  highlighted  NCO 
concepts. 

Note:  UK  (Graham  Mathieson)  use  of  the  term  “Full  spectrum  analysis”  in  his 
title  refers  to  the  inclusion  of  a  wide  spectrum  of  “human  factors.” 
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Legend:  The  color  of  text  in  each  box  was  changed  to  either  green,  orange  (italic)  or  red 
(underlined)  as  per  the  definitions  on  the  next  chart  for  the  “state  of  the  OA  practice” 
assessments.  The  purpose  of  the  color  coding  was  to  present  a  general  assessment  of  the 
state  of  the  OA  practice  overall  in  to  support  the  key  NCO  concepts  identified  in  the 
framework. 

Describe  how  each  high-level  capability  (and  corresponding  high-level  measure)  fits 
into  the  domains.  How  are  information  services  provided,  etc?  Right  now,  the 
framework  is  not  fully  populated  to  support  analyses.  Sensemaking  terminology  need  a 
lot  more  consideration. 
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Color  Meaning 


insHS 

Mtlltary  Operations  Research  Society 


•  Green  -  know  how  to  do.  Have  to  adapt  models.  No  new  research. 

•  Orange  -  know  how  to  do  but  would  be  serious  model 
development  Some  research  involved.  May  be  able  to  adapt 
methods  including  from  other  disciplines. 

•  Red  -  Significant  research  and  major  modeling  development,  (we 
don’t  have  a  clue) 
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Assessment  of  Framework  and  Tenets 

1  Information  Value  Added  | 

Sources  Services  \ 

II  Degree  of  Networking;  I  -  .  1  Net  Readiness  of  Nodes  I. 'll 

-I  Degree  of  Information  “Share-ability”  '  \  , 

. . . i i _  ;  : 

Qual  ity  of  Individual  information 

. :  of  Shared  ^formation '  r'  - ' 

;  A  rpbustly  networked  fi 

1.  Physical  Domain:  Resolution  of  th 

*  Today’s  Physical  network:  (P\ 

•  Information  sources  fexceot  I 

2.  Information  Domain 

•  Quality  of  organic  infoiTnatio! 

•  Quality  of  net  working 

•  Degree  of  networking 

•  Net  readiness  {mformarn 

*  Degree  of  infonnation  "'share 

♦  Quality  of  indlvlduai  iaformat 

’  Degree  of  shared  infoiwarloti 

)rce  improves  information  sharing 

2  force  (architectures) 

Better  iob  rnaoDine  architectures  to  OA 

■iUMINT)  - - - 

1  {metrics  well  defined) 

Families  of  models 
Communities  of  interest 
Dynamic  bandwidth  allocation 
ion  ffusion)  Dynamic  adaptive  architectures 

Spectrum  of  countermeasures 

This  chart  exemplifies  how  the  WG  systematically  decomposed  the  top-level 

Framework  model  and  recorded  their  assessments  as  to  the  state  of  the  OA  practice. 
In  doing  so,  the  group  also  considered  the  7  tenets  of  NCO  in  relation  to  the 
framework.  Discussion  for  this  example  led  to  distinguishing  what  OA  capabilities 
can  be  done  reasonably  well  from  those  OA  capabilities  that  are  deficient. 

Physical  Domain:  Resolution  of  the  force  (architectures) 

-  Today’s  Physical  network  (Future) 

-  Information  sources  lexcept  HUMINT) 

-  Better  job  mapping  architectures  to  OA 

Information  Domain 

Quality  of  organic  information  (metrics  well  defined) 

Quality  of  networking 

Degree  of  networking 

Net  readiness  (mformation  assurance) 

Degree  of  in  format  ion  “share-ability  ” 

Quality  of  individual  information  (fusion) 

Degree  of  shared  information  (multi-level  security  sharing  across  coalitions) 
Families  of  models 
Communities  of  interest 
Dynamic  bandwidth  allocation 
Dynamic  adaptive  architectures 
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State  of  Health  -  OA  of  7  NCO  Tenets  fil©7?15> 

Military  Operations  Research  Society 


•  Robustly  networked  force  Y 

(Dynamic  Networks) 

•  Information  sharing  Y 

(Information  Assurance) 

•  Collaboration  R 

•  Quality  of  information  G 

•  Shared  situational  awareness  Y 

•  Self-synchronization  R 

•  Sustainability  and  speed  of  command  R 


Info  sharing:  The  community  has  ideas  today  on  how  information  moves  around 
the  battlespace,  but  the  issues  of  Information  Assurance  (lA)  precipitated  the 
evaluation  of  the  “state  of  the  practice”  to  be  degraded  from  green  to  yellow. 

Understanding  and  modeling  of  collaboration  —  especially  at  the  social  level,  is 
very  weak. 

Cross-cultural  issues  (across  agencies,  NGOs,  etc.)  will  require  “technology 
transfer”  to  be  improved. 
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State  of  Health  -  Domains 


HHitary  Operations  Research  Society 


•  Physical  Y 

•  Information  Y 

•  Cognitive  R 

•  Social  R 


The  state  of  health  for  OA  in  support  of  NCO  is  weak  overall.  Even  at  the 
physical  level  of  modeling  and  analysis,  the  fact  that  the  OA  community 
continues  to  focus  on  the  warfare  (combat)  portion  of  NCO  and  not  on  the  larger 
set  of  military  missions  (e.g.,  OOTW)  is  seen  in  the  scarcity  of  methods  for  these 
situations.  The  physical  and  information  domains  were  also  rated  yellow  by  the 
group  because  OA  practitioners  do  not  generally  embrace  the  complexity  and 
emergence  involved  in  NCO  and  continue  to  attempt  to  reduce  the  problem  to 
linear  dynamics. 

Overall,  the  MORS  OA  community  is  uncomfortable  with  expanding  their 
analysis  into  the  cognitive  and  social  levels.  Scientific  approaches  of  the  past 
that  addressed  the  physical  domain  well  seemingly  don’t  represent  these  other 
domains  well.  The  OA  community  needs  to  consider  methods  from  other 
disciplines.  Medicine,  genetics,  sociology,  political  science  and  cultural 
anthropology  fields  are  examples  of  fields  with  increasing  relevance  to  future 
OA. 
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Problem  Solving:  Analysts  and  the  decision  makers  they  support  often  don’t 
know  what  the  relevant  problem  and  questions  are  right  now.  Today  there  are 
different  facets  of  problems  we  need  to  deal  with;  how  do  we  address  these 
methodologically,  how  do  we  address  these  culturally  and  resource  wise.  There 
are  not  unlimited  resources  and  there  are  cultural  barriers  to  approaching  this 
new  class  of  problems. 

Scenarios:  Know  how  to  do,  but  the  OA  community  hasn’t  begun  to  explore  yet. 
OA  could  help  the  process  of  defining  scenarios  by  asking  the  right  questions. 

Tools  (deferred  this  block  to  the  tools  WG). 

Data:  If  we  thought  about  data  differently,  we  could  exploit  resources  that 
already  exist.  We  don’t  take  a  robust  enough  approach. 

Reporting:  In  many  cases,  OA  analysts  are  operating  in  a  realm  where  decision 
making  tempo  is  shorter  than  we  can  address  the  questions  with  analysis. 
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State  of  the  Practice  Iil©3ll® 

•The  maturity  of  an  “NCO  OA  process”  is 
consistent  with  the  state  of  the 
understanding  of  NCO.. ..immature 

♦  Current  tools  and  practices  are  understandably  grounded  in  what 
we  know  and  the  decision  process  we  support. 

♦  Some  of  that  foundation  is  essential 


WG  2  participant  comments  on  state 
of  the  practice  are  in  Backup  Material 


Need  to  ensure  that  there  is  value  in  what  we  have  been  doing  to  what  we  need  to 
be  doing.  (Need  to  discern  what  to  shed  and  what  to  keep.) 
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Top  Recommendations  mms 

Military  Operations  Research  Society 

•  MORS  should  undertake  an  effort  to  extend  the 
Analyst’s  Handbook  to  address  NCO  OA  Issues. 

•  MORS  Sponsors  should  encourage  continued 
training  and  education  related  to  NCO  Concepts, 
and  supporting  analyses  (models,  simulations, 
experiments) 

•  Analysts  should  examine  what  knowledge  and 
processes  can  be  drawn  from  non-traditional 
disciplines 


WG  2  participant  recommendations 
are  in  Backup  Material 


Recommendation  1 :  Given  all  the  on-going  changes,  how  will  we  update  this 
handbook?  Will  be  soliciting  input  regarding  this. 

Recommendation  2;  Training  and  education  was  seen  to  be  critically  important 
to  transformation  overall,  and  specifically  to  the  OA  practice  in  support  of  NCO. 
This  was  the  same  for  government,  military  and  industry;  and  was  felt  to  be  the 
same  internationally  as  well  as  in  the  US. 

Recommendation  3:  OA  needs  to  get  back  to  its  multi-disciplinary  roots.  Need 
to  seek  out  social  sciences  as  well  as  the  hard  sciences  and  engineering  fields  for 
ways  to  abstract  and  approach  squishy,  and  seemingly  intractable  problems. 
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State  of  the  Practice  (1/2)  mms 

Military  Operations  Research  Society 

1.  Trying  to  teach  “old  dogs  new  tricks.”  Modifying  legacy  models  that  are  not 
information  centric  to  account  for  information  in  the  modeling  and  simulation 
process.  If  and  when  the  old  models  can  be  modified,  the  outcomes  can  only  be 
measured  in  terms  of  attrition. 

2.  “We”  will  not  achieve  revolutionary  results  using  the  same  analysts.  As  we 
design  these  inherently  complex  systems,  people  with  fixed  mental  models  of 
operations  will  not  perceive  the  non-linear  results.  The  majority  of  the  lead 
analysts  in  the  workshop  have  20+  years  experience  in  OA. 

3.  Current  tools  and  practices  do  not  address  or  capture  collaboration,  self¬ 
synchronization  or  agility  -  key  characteristics  of  NCO. 

4.  The  community  continues  to  use  tools  and  assumptions  which  appear  to  be 
invalid  in  today’s  warfighting  environment,  and  then  conduct  verification  and 
validation  of  new  tools  by  comparing  outputs  with  existing  tools. 

5.  Practice  needs  to  be  matured  in  terms  of  analyzing  the  “entry-fee”  to  NCO,  i.e., 
assessing  infostructure  in  terms  of  enabling  NCO. 


6.  M&S  development  is  taking  too  long  and  not  amenable  for  quick  turn  around  or 
cost  effective  strategic  analysis,  i.e.,  JWARS.  Network  M&S  tools  for  NCO  are 


These  two  slides  contain  the  input  of  the  28  working  group  members  most 
important  individual  observations  about  the  state  of  the  practice. 
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State  of  the  Practice  (2/2)  ni®H3 

Military  Operations  Research  Society 

7.  The  OA  community  is  not  focusing  on  the  full  breadth  of  military  missions  -  but  rather, 
it  is  continuing  to  focus  on  analyzing  what  we  already  know^  i.e.,  the  “W”  (or  workforce) 
part  0f“O” 

8.  There  also  is  a  scarcity  of  research  to  model  non-warfare  aspects  of  NCO. 

9.  SoS  architecture  analysis  is  pivotal  to  assessing  NCO.  Yet  the  classical  OA  pyramid  of 
modeling  and  simulation,  the  OA  process  (as  described  on  day-1)  and  even  the  detailed 
process  described  in  the  recent  book  NATO  Code  of  Best  Practices  for  Assessment  (DoD 
CCRP,  2002)  —  do  not  explicitly  address  the  role  of  architecture  analysis  in  the 
assessment  of  Systems  of  Systems  (SoS).  This  lack  of  process  clarity  is  evident  as  the 
OA  community  is  challenged  more  and  more  with  complex,  SoS  solutions  (DOTMLPF) 
for  meeting  the  capability  needs  for  NCO. 

10.  NCO,  CBA  and  EBO  together  create  a  large  analysis  space  and  scenario  space.  The 
amount  of  funding  and  other  resources  for  analysis  doesn’t  increase  accordingly.  What 
must  change  to  deal  with  this  large  ‘space’? 

11.  The  use  of  NCW  consideration  in  OA  is  embryonic.  The  US  Army  FCS  and  Navy 
FORCEnet  activities  are  initial  efforts. 

12.  (Shared)  awareness/understanding  is  not  dominated  by  (shared)  situation  information 
but  by  a  combination,  in  roughly  equal  parts,  of  (shared)  history,  experiences  and 
situation  information. 
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Short-Term  Recommendations  (1/2)  mms 

Mltltary  Operations  Research  Society 

1.  Analysts  need  to  portray  their  subjects  of  analysis  in  terms  of  applicable 
architectures.  This  will  facilitate  assessing  the  degree  of  their  impact  on 
capabilities. 

2.  Use  USAF  -  AFSAA  SEAS  tool  in  conjunction  with  JWARS. 

3.  Develop  an  OA  community  strategy  document  that  includes  priority 
efforts. 

4.  Invest  in  tool  development  for  assessing  C4ISR  systems. 

5.  Most  military  and  civilian  ORSA  lack  adequate  training  about  knowledge 
and  data  management.  This  will  be  chronic  if  not  fatal  for  analysis  of 
info-based  NCO.  We  need  to  institutionalize  the  requirement  that 
ORSA’s  are  managers  of  data,  not  just  users.  Address  this  problem  in  the 
short-term  with  training. 

6.  Need  to  prioritize  efforts  in  respect  to  NCO  representation  and 
measurement  for  all  the  different  uses  of  analysis,  e.g.,  experimentation, 
training,  resource  allocation,  etc. 

7.  Produce  NCO  analysis  that  is  75%  correct  but  timely,  plenty  accurate  for 
inferring  strategic  insights. 


These  slides  contain  a  set  of  individual  short  term  recommendations  identified  by 
all  of  the  group  members  regarding  OA  for  NCO.  These  were  synthesized,  and 
in  some  cases  deferred  to  other  groups  because  of  the  focus  of  the  group. 
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Short-Term  Recommendations  (2/2) 


mms 

MUltary  Operations  Research  Society 


8.  Update  and  expand  the  OA  process  to  reflect  where/how  architecture 
analysis  and  the  more  classical  pyramid  of  M&S  fit  together  in  a 
evolutionary  concept  development  or  exploratory  analysis  of  NCO. 

9.  Document  lessons  learned  from  initial  NCW  studies. 
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Lons-Term  Recommendations 

^  MIHtary  Operations  Research  Society 

1 .  Stress  that  new  modeling  efforts  should  be  built  on  an  information 
paradigm.  Measurable  outcomes  should  consider  “soft  factors”  such  as  C2, 
leadership,  will  to  fight,  etc. 

2.  Incorporate  agent-based  simulations  in  M&S  tools. 

3.  Ensure  OA  data  collection  requirements  are  included  in  the  on-going 
development  of  the  Joint  National  Training  Capability  (JNTC). 

4.  Educate  military  analysts  on  the  NCO  Conceptual  Framework. 

5.  Formulate  a  coherent  plan  for  addressing  collective  NCO  analysis 
requirements.  Need  the  plan  to  coordinate  efforts,  ensure  consistency  in 
NCO  representation,  and  to  ensure  consistent  measurement  of  effects. 

6.  Consider  modifying  the  JWARS  -  Operational  Requirements  Document 
(ORD)  to  specify  requirements  to  model  NCW, 


These  slides  contain  a  set  of  individual  long  term  recommendations  identified  by 
all  of  the  group  members  regarding  OA  for  NCO.  These  were  synthesized,  and 
in  some  cases  deferred  to  other  groups  because  of  the  focus  of  the  group. 
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Will  describe  how  each  high-level  capability  (and  corresponding  high-level  measure) 
fits  into  the  domains. 
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Building  the  Framework  on  the  Tenets  ofNCO 


Information 

Sources 


Value  Added 
Services 


Force 


C2 


Effectors 


Quality  of  Individual  Sensemaking 

Lack  ability  to  implement  theories  of  individual  sensemaking  in  cognitive  models  to  capture  agent 
interaction 

Currently  implemented  hy  rule  sets 
Criiicady  dependent  on  context 

Sensemaking  may  not  be  mature  enenigh  to  support  decision  niakhtg 
Approaches  from  other  disciplines  thai  may  be  useful 

Currentfv  embed  sensemaking  in  sinudations  using  ruJ es/ k  t  an  dud oop/ algorithm  for  a  decision 

Criteria  required  by  decision  makers  to  address  sub-steps  or  just  analyze  decisions  (models  vs.  war- 
games)? 

Should  we  consider  Sensemaking  vs.  Decision  Making  as  part  of  the  framework  (per  DoD  Joint  Ops 
concept  NOV  2003)7 

Need  more  research  on  Cognitive  modeling  (to  address  the  non-automated  processes) 

Modeling  situational  awareness 
Analysis  of  results  of  Individual  Scnscinaking 
Quality  of  Interactions 
Must  model  modes  of  interaction 
Clarification  on  the  context 
Address  types  of  networks  (formal,  informal,  etc.) 

•  Degree  of  Shared  Sensemaking 

Corollary  to  individual  sensemakine  but  allows  for  different  outcomes  (emerging  group  behavior) 
Difficult  to  understand  the  group  dynamics  to  implement  in  simulations 
Very  context  sensitive 

Significantly  increased  interactions  within  the  gi'Oim  (randomness/complexitv) 

Bound  the  problem  by  Jointness  through  US-onlv  to  Coalition  (Joint  modeling  is  an  example) 

Need  increased  input  from  sister  services  to  sensitivity 

Analysis  of  results  of  Shared  Sensemaking 
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Degree  of  Decision  Synchronization 

Evaluating  States  vs.  outcome  (robustness,  agility,  etc.)  must  be  supported  by  MOEs 
Performance  vs.  scenario  space 

Models  are  available  for  Decision  Synchronization  (but  need  to  account  for  system 
dynamics) 

Not  modeling  self-synchronization  ('dynamics  of  decision  to  action  synchronization) 

Agent-based  models  are  beginning  to  develop  this  capability 

Local  vs.  Central  Control  is  an  issue  in  agent-based  models  (increased  degrees  of 
freedom)  with  significant  results. 

Commercial  examples  of  cognitive  modeling  (large  discount  stores,  grocery  stores,  etc.) 
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Building  the  Framework  on  the  Tenets  ofNCO 
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Military  Operations  Research  Society 


Operations  Analysis  Support  to 
Network  Centric  Operations 

Military  Operations  Research 
Society  (MORS)  Workshop 

WG  #3:  Assessment  Toois 
Outbrief 

This  group  included  24  members,  including  representatives  from  5  different 
countries  (US,  UK,  The  Netherlands,  Denmark  and  Israel): 


65 


WG  3  -  Assessment  Tools 


Military  Operatloiu  Research  Society 


•State  of  the  Practice 

♦  Level  -  Inconsistent  Use 

♦  Applications/Examples 

Use  of  JWARS/GCAM/NSS  in  Navy’s  FORCEnet  analysis 
Quantifying  Information  in  Availability  for  Situationai  Awareness  using 
0TB 

Use  of  CASTFOREM  in  FCS  AOA 

^  UK’s  balancing  investment  between  equipment  and  network  options 


Much  of  the  discussion  was  focused  on  the  state  of  the  practice  of  assessment  tools  for  NCO. 
Most  of  the  presentations  during  the  second  day  were  on  models  and  simulations.  It  was  the 
groups  conclusion  that  in  the  context  of  NCO,  the  tools  were  used  infrequently.  During  the 
second  day  of  the  workshop,  the  working  group  listened  to  presentations  on  some  current  NCO 
tools  in  use.  It  was  the  conclusion  of  the  group  that  the  tools  were  used  infrequently  and  can  be 
attributed  to  single  service  development,  portrayal  etc.  Also,  the  tools  are  designed  for  a 
specific  purpose,  and  with  NCO  being  a  transformational  concept,  the  tool  set  needs  to  be 
developed  in  order  to  be  sure  we  are  assessing  NCO  correctly. 

The  working  group  listened  to  four  presentations  on  the  second  day  of  the  workshop.  During 
the  first  working  session,  we  participated  in  the  JWARS  joint  session  in  the  auditorium  with 
Working  Groups  1  and  2.  During  the  second  session.  Dr.  Ben  Taylor  and  Lynda  Sharp 
presented  the  “UK  High-level  Campaign  models  and  NEC  related  Studies.”  Doug  Mackey 
then  presented  on  CASTFOREM  followed  by  LTC  Stephen  Riese’s  presentation  on 
“Quantifying  Information  Availability  for  Situation  Awareness”  during  the  third  working 
session.  The  final  session  of  the  day  was  used  to  discuss  these  presentations  and  discuss 
questions  pertaining  the  charge  to  the  working  group. 

Some  important  discussions  took  place  on  the  final  day  of  the  workshop.  We  were  picking 
away  at  the  problem  of  identifying  the  state  of  the  practice  of  assessment  tools  for  NCO. 
Initially  some  discussion  accepted  that  the  physical  modeling  was  pretty  good,  though  it  is 
incapable  of  illustrating  effects,  as  illustrated  in  EBO  (on  a  smaller  scale).  The  simulations  can 
show  the  loss  of  a  power  grid  or  damage  on  a  railway,  but  it  cannot  illustrate  how  these  losses 
really  effect  the  accomplishment  of  the  overall  goals  of  the  operation. 


66 


fFG  3  -  Assessment  Tools 
•  State  of  the  Practice  (by  domains) 


Domain 

Assessment 

Comment 

Physical 

GREEN 

•  Many  models  exist  for  the  physical  domain  (movement,  attrition,  etc.)  at  all  levels 
of  aggregation  and  fidelity. 

Information 

YELLOW 

•  Concentrates  on  how  to  deliver  information  superiority. 

•  Sensor  and  communications  modelling  are  adequate  if  not  modelled  as  widely  as 
physical  domain. 

•  Combat  models  exist  at  all  levels  from  campaign  to  mission  that  can  demonstrate 
the  sensitivity  of  attrition-based  combat  to  improved  information  availability. 

Cognitive 

ORANGE 

•  Link  between  information  and  decisions  is  non  trivial  and  that  information 
superiority  does  not  imply  decision  superiority. 

•  Models  that  replicate  human  decision  making  (beyond  rule  bases)  do  exist,  but  are 
not  widely  embedded  in  combat  simulations 

•  Extensive  work  required  to  calibrate  their  behaviour  to  real  human  decision  makers 

•  Requires  collection  of  new  kinds  of  data  from  experimentation  and  real  operations. 

Social 

RED 

•  Flexible  networks  of  collaborating  individuals  work  remains  mostly  within  the 
research  environment 

•  Some  work  exists  on  emergent  properties  of  groups  in  adaptive  agent  based  models 
(e.g.  Project  Albert). 

•  The  military  OA  analysts  need  to  engage/include  social  scientists  and  both 
individual  and  group  psychologists. 

The  working  group  assessed  the  “State  of  the  Art”  in  relation  to  each  of  the  four  domains 
associated  with  Network  Centric  Warfare.  The  results  of  the  discussion  are  outlined  as 
follows: 

Physical  Domain:  Traditional  combat  models  encompass  the  physical  domain; 
movement,  attrition,  logistics,  etc  are  understood  and  captured  in  the  existing  toolset. 

This  is  not  a  priority  area  for  attention.  Many  models  exist  for  the  physical  domain  at  all 
levels  of  aggregation  and  fidelity. 

Information  Domain:  Modelling  here  concentrates  on  how  to  deliver  information 
superiority.  Sensor  modelling  and  communications  modelling  are  adequate  if  not 
modelled  as  widely  as  physical  domain.  Combat  models  exist  at  all  levels  from  campaign 
to  mission  that  can  demonstrate  the  sensitivity  of  attritional  combat  to  improved 
information  availability. 

Cognitive  Domain:  Research  has  shown  (Riese  et  al  and  others)  that  the  link  between 
information  and  decisions  is  non  trivial  and  that  information  superiority  does  not  imply 
decision  superiority.  Models  that  replicate  human  decision  making  (other  than  rule  bases 
-  which  can  be  shown  not  to  represent  the  human  thought  process)  do  exist,  but  are  not 
widely  embedded  in  combat  simulations.  Where  such  models  do  exist  extensive  work 
will  be  required  to  calibrate  their  behaviour  to  real  human  decision  makers.  This  will 
require  the  collection  of  new  kinds  of  data  from  experimentation  and  real  operations. 

Social  Domain:  The  way  in  which  flexible  networks  of  collaborating  individuals  work 
(and  how  they  can  be  stopped  from  working)  is  an  area  that  remains  mostly  within  the 
research  environment.  Some  work  exists  to  understand  the  emergent  properties  of  groups 
in  adaptive  agent  based  models  (e.g.  Project  Albert).  The  military  OA  analysts  need  to 
engage  with  a  broader  range  of  SMEs,  including  social  scientists  and  both  individual  and 
group  psychologists. 
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fVG  3  -  Assessment  Tools 


Military  Operations  Research  Sodety 


•Key  Issues,  Shortfalls  and  Gaps 

♦  Key  Issues 

Relationship  between  EBO,  10  and  NCO 

-  If  value  of  NCO  in  delivering  EBO  can't  be  demonstrated  then  NCO 
is  not  being  fully  assessed 

~  10  is  a  contributor  to  EBO  and  hence  NCO  must  be  able  to  deliver  10 
^  Consistency  of  NCO  tenets/meanings  at  different  levels  of  command 

-  Impact  of  degree  of  networking  -  who  should  be  connected  to  whom 
and  how  much  data  sharing 

^  Relationship  between  quality  of  decisions  and  the  information  available 

-Link  between  increased  data,  information  and  knowledge  resulting  in 
a  decision  or  action  is  a  complex  issue 
Risk/vulnerability  associated  with  reliance  on  NCO  -  assess  reliance  on 
our  networks 


Some  issues  and  shortfalls  that  were  identified  by  the  working  group  are  summarized  below.  It  should  be  noted 
that  it  is  difficult  to  differentiate  between  possible  shortfalls  or  reservations  with  the  concept  of  NCO  rather 
Aan  the  tools  available  to  assess  NCO. 

Relationship  between  concepts: 

-  Effects  Based  Operations  is  a  philosophy  of  conflict  designed  to  defeat  an  enemy’s  ability  and/or 
willingness  to  fight  by  a  broader  set  of  means  than  the  physical  destruction  of  combat  power 

-  Network  Centric  Operations  is  a  way  in  which  military  forces  will  conduct  operations  to  deliver  EBO 

-  Digitisation  is  the  introduction  of  the  technology  required  to  enable  NCO 

-  The  consequence  of  the  above  is  that  the  tools  required  to  assess  NCO  are  a  subset  of  those  required  to 
assess  EBO.  If  the  value  of  NCO  in  delivering  EBO  cannot  be  demonstrated  then  NCO  is  not  being  fully 
assessed. 

-  Effects  Based  Operations  deliver  success  by  attacking  an  enemy  in  ways  beyond  the  attritional.  Tools  to 
assess  EBO,  anci  hence  NCO,  must  address  a  wider  set  of  outputs  than  attrition  and  hence  traditional 
combat  modellers  are  unlikely  to  have  understanding  of  the  issues  required  to  model  EBO  or  NCO 
properly.  INSIGHT:  Model/Tool  development  projects  must  include  access  to  relevant  expertise  in  social 
and  cognitive  domains. 

NCO  means  different  things  to  different  people  at  different  levels  of  command.  It  is  centered  around  “what 
can  NCO  do  for  me”  and  the  answer  being  a  wide  variety  of  issues,  makes  it  difficult  to  assess  the  effects  of 
NCO  throughout  a  force.  Issues  such  as  “how  much  networking”  is  necessary  were  discussed  though  no 
conclusion  was  brought  to  fiuition.  The  sergeant  on  the  field  may  not  need  as  much  networking  as  the 
command  centers. 

Discussions  led  to  the  revelation  that  there  are  few  tools  to  examine  the  relationship  between  the  quality  of  the 
decisions  and  the  amount  of  data  that  is  available  to  the  decision  maker.  Stephen  Riese’s  presentation 
highlighted  the  fact  that  some  commanders,  even  though  they  have  high  situation  awareness,  count  on  intuition 
rather  than  available  data.  The  issues  relating  to  the  data,  information,  and  knowledge  and  the  results  is  not 
captured  with  the  current  tool  set. 

Transforming  a  force  to  Network  Centric  Operations  raises  serious  concerns  about  the  forces  reliance  on  the 
network.  Tools  need  to  be  developed  to  assess  the  vulnerabilities  related  to  this  reliance  on  the  network.  Again, 
this  can  be  an  assessment  or  concern  about  the  theory  of  NCO  rather  than  the  tools  used  to  assess  NCO 
concepts  and  operations. 
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fVG  3  -  Assessment  Tools 


Hilitary  Operations  Research  Sodety 


•Key  Issues,  Shortfalls  and  Gaps  (cont.) 

♦  Key  Shortfalls/Gaps 

Data/protocols  -  sources,  classification,  accessibility,  releasability 

-  Need  to  include  social  scientist  in  the  data  collection  during 
wargamming  and  exercises 

Ability  to  capture  the  human  dimension/cognitive  aspects 

-  Factors  that  address  the  quality  of  decision  making 

-  Capture  information  needed  by  the  commander  -  business 
processes 

Need  vulnerability  analysis  tools 


A  huge  shortfall  in  tool  creation  and  application  is  data  availability.  Data  for 
creating  and  populating  models  is  difficult  to  acquire.  The  sources,  classification, 
accessibility,  and  releaseability  issues  hinder  the  productiveness  of  the  tools. 
Also,  results  that  are  classified  are  not  as  beneficial  to  the  community  than  those 
which  are  easily  distributed.  This  is  crucial  to  get  community  buy  in,  and  to  test 
the  theory  of  NCO  rigorously  prior  to  commitment  to  transform  the  force. 

Data  need  to  be  gathered  and  collected  by  social  scientists  among  others.  NCO  is 
Human  Centric,  and  those  who  study  humans  need  to  assist  in  gathering  the  data 
because  they  may  recognize  behaviors  and  characteristics  that  engineers  and 
analysts  may  not  recognize. 

The  human  dimensions  of  NCO  currently  are  not  captured  and  explored.  Some 
issues  that  were  addressed  earlier  in  the  brief  included  what  information  is 
necessary  to  improve  decision  making  and  assessing  how  much  information  is 
needed  by  the  commander  to  make  a  good  decision.  These  questions  essentially 
highlight  the  business  processes  that  are  involved. 
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fVG  3  -  Assessment  Tools 


li1®R3 

Military  Operations  Research  Sodety 


•  Key  Issues,  Shortfalls  and  Gaps  (cont.) 

♦  Key  Issues 

Models/Tools  must  include  access  to  relevant  aspects  of  social  and  cognitive 
domains 


-  Tools  to  assess  EBO,  and  hence  NCO,  must  address  a  wider  set  of  outputs 
than  attrition 

^  Need  to  determine  what  level  of  modeling  is  appropriate  for  NCO  analysis 

-  Do  campaign  models  capture  the  right  types  of  decisions  and  decision 
making  process? 

-  Need  for  a  family  of  models  from  engineering  to  campaign  with  appropriate 
connectivity 

^  Models  must  be  able  to  capture  the  relationship  issues  associated  with  coalition 
warfare 

-  Different  levels  of  “network  compliance”  in  both  hardware  and 
training/doctrine 

Models  must  focus  on  the  appropriate  NCO  issues  -  doing  things  faster  vs. 
doing  better  things 


(Continued  from  Previous  page.) 

Addressing  the  Human  issues  is  necessary  to  conduct  proper  assessment  of  NCO 
therefore  assessments  and  development  MUST  include  professions  such  as  social 
scientists  in  order  to  assess  the  concepts  fully.  The  term  Network  Centric 
Operations  is  sometimes  misleading.  At  a  closer  look,  it  is  really  Human  Centric 
and  Network  Enabled. 

The  toolset  must  be  able  to  address  the  high-level  ‘so-what’  questions  at  the 
campaign  and  national  force  structure  level.  Therefore  tools  are  required  that  can 
work  at  this  level.  Since  we  have  little  or  no  NCO  data  with  which  to  calibrate 
high-level  models  the  tool  set  must  include  lower  level  models. 

If  coalition  operations  are  to  be  undertaken  in  the  future  then  (at  least)  campaign 
level  models  must  be  able  to  capture  the  issues  of  the  relationship  with  allied 
forces  having  different  levels  of  ‘network  compliance’  in  both  their  hardware  and 
training/doctrine. 

The  models  need  to  focus  on  the  correct  aspects  of  NCO.  What  makes  a  network 
centric  force  better  than  a  platform  centric  force?  How  are  the  people  able  to 
innovate  and  devise  new  ways  of  doing  things? 
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fVG  3  -  Assessment  Tools 


Military  Operations  Research  Society 


•Future  Challenges 

♦  Coalition  warfare  analysis 

Interoperability,  data/information  sharing,  ambiguity  in  communications 

♦  Development  of  a  set  of  models  or  interoperability  between  models 

♦  Balance  between  representation  of  devolved  responsibility  and 
aggregated  performance 

♦  Need  to  capture  all  aspects  of  NCO  to  be  able  to  represent  future 
forces 

♦  Models  need  to  adapt  to  changing  concepts  -  e.g.  dynamically  task 
organize  vs.  rigid  inputted  organization  structure 


Coalition  warfare  analysis  is  lacking  in  today's  assessment  tools.  In  the  US,  analysis 
of  joint  operations  is  making  some  headway,  but  most  operations  in  the  future  will 
be  composed  of  coalition  forces.  Throughout  recent  history,  the  ability  to  share 
information  in  a  coalition  environment  has  been  difficult  though  necessary.  Other 
issues  that  need  to  be  addressed  in  the  coalition  environment  are  the  language 
barrier  and  systems  interoperability. 

In  order  to  represent  NCO  accurately,  a  new  set  of  models  or  a  new  way  of 
interfacing  between  them  needs  to  be  created.  The  amount  of  granularity  necessary 
to  represent  the  lower  level  effects  and  the  scaling  to  represent  the  operational/ 
strategic  outcomes  require  a  set  of  tools  that  are  interoperable.  After  all  why  would 
NCO  be  a  new  way  of  thinking  if  we  already  have  models  to  represent  it? 

NCO  assessment  tools  will  require  detailed  performance  of  the  lower  levels  of 
command  (as  illustrated  in  Power  to  the  Edge).  This  devolved  responsibility  needs 
to  be  paralleled  with  an  aggregate  level  model  to  highlight  the  greater  force 
performance  due  to  pushing  the  new  capability  of  the  lower  level  units. 

In  order  to  represent  NCO  accurately  and  conduct  accurate  NCO  analysis,  all 
aspects  of  NCO  need  to  be  represented.  This  means  networking  not  only  two 
battalions  but  all  actors  involved  in  the  operation  including  reachback  capabilities. 
This  is  the  essence  of  NCO,  and  the  capabilities  cannot  be  explored  if  they  are  not 
represented.  In  essence,  the  tools  need  to  be  able  to  represent  the  future  forces. 

Essentially,  the  tools  that  need  to  be  developed  need  to  be  able  to  adapt  to  changing 
and  emerging  concepts.  This  is  necessary  for  assessment  of  new  ways  of  doing 
things  vs.  the  old  way  of  doing  things. 
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fVG  3  -  Assessment  Tools 


li16>H3 

Military  Operatioru  Research  Sodety 


•Extent  of  Collaboration 

♦  Among  (and  within)  the  Services 

Level  -  Inconsistent 
Examples 

-  Service  specific  NCO  analysis  like  PCS  AOA  and  Navy’s  FORCEnet  analysis 

-  Recognized  need  for  less  emphasis  on  service  specific  models 

♦  With  the  Coalition  Forces 

Level  -  Infrequent  and  inconsistent 
Examples 

-  UK/US  ground  component  Interoperability  study 

-  TTCP  JSA  TP-3's  future  concepts  focus  area 

♦  With  the  US  Agencies 

Level  “  Infrequent 
Examples  -  potential  exists 

-  Homeland  defense 


The  levels  of  collaboration  among  and  within  the  services  is  inconsistent. 
Currently  the  NCO  analysis  that  is  being  conducted  is  service  specific,  though  it 
is  recognized  that  to  be  taken  seriously  joint  assessment  is  needed. 

Collaboration  with  the  coalition  forces  is  infrequent  and  inconsistent.  The 
workgroup  could  not  identify  many  examples  of  coalition  collaboration.  Thou^ 
the  UKTUS  ground  component  interoperability  study  and  the  TTCP  JSA  TP-3’s 
future  concepts  group  came  to  mind. 

With  US  agencies,  collaboration  was  infrequent  though  potential  was  seen  to 
interact  with  the  Department  of  Homeland  Security.  Representatives  from  this 
area  were  not  present  and  the  group  felt  it  was  inappropriate  to  make 
assumptions. 
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WG  3  -  Assessment  Tools 


Hltitary  Operattons  Research  Sodety 


•  Recommendations 

♦  Specific  to  WG  area 

Initiate  activity  to  identify  expertise  in  social  network  analysis  tools  and  integrate 
that  expertise  Into  current  and  future  tools 

Initiate  activity  to  incorporate  appropriate  representation  of  cognitive  processes 
In  current  and  future  tools 

^  Develop  guidelines  or  principles  for  establishment  of  a  set  of  appropriately 
connectivity  models 

Data,  Data,  Data  -  issues  of  accessibility,  releasabllity  and  classification  must 
be  resolved 

♦  Overall  to  OA  Support  to  A/CO 

Identify  the  linkages  between  NCO,  EBO  and  10 

Initiate  NCO-related  studies  involving  Coalition  partners  to  improve 

understanding  of  coalition  issues 

♦  Other  Comments 

Hold  all  January  Workshops  in  Key  West 


The  working  group  formulated  a  set  of  recommendations. 

In  order  to  address  the  issue  of  assessing  the  cognitive  areas  of  NCO,  social  scientists  and  their 
tools  need  to  be  included  into  the  assessment.  The  views  of  the  social  scientists  and  the 
engineers  are  different  and  therefore  data  collection,  tool  application,  assessment  tools  etc.  will 
need  to  keep  these  different  views  in  mind. 

The  current  tool  set  does  not  capture  the  relevant  cognitive  aspects  associated  with  NCO.  This 
is  partially  due  to  the  limited  knowledge  (or  over  abundance  of  opposing  views)  of  cognitive 
functions.  Therefore  the  appropriate  representation  of  cognitive  processes  in  current  and  future 
tools  needs  to  be  pursued. 

The  modeling  associated  with  NCO  needs  guidelines.  These  guidelines  will  help  create  models 
that  can  be  interconnected  and  compatible  so  all  the  aspects  of  NCO  can  be  represented. 

Data  is  a  huge  issue.  Continuing  issues  with  data  accessibility,  releasability  and  classification 
need  to  be  resolved.  Also,  in  NCO,  we  need  to  be  sure  that  we  are  collecting  the  right  data 
necessary  to  conduct  proper  assessment.  Industrial  age  data  collection  may  not  be  the  proper 
method  for  collecting  data  in  the  information  age. 

Overall  Recommendations: 

The  linkages  between  and  among  the  emerging  concepts  of  NCO,  EBO  and  10  should  be 
identified  (an  attempt  was  made  earlier  in  this  brief) 

In  the  future,  coalitions  will  be  commonplace.  Therefore  it  is  imperative  that  coalition  studies 
of  NCO  be  initiated.  As  it  stands,  coalition  partners  are  working  toward  network  centric 
concepts,  though  a  greater  understanding  of  research  and  NCO  related  activities  is  necessary. 
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...  Co-evolution  of  Key  Elements 


Today 

. . . 


Transformation  is  clearly  about  moving  from  an  industrial  age  force  to  an 
information  age  force  -  we  had  no  dissent  on  that  point. 

WG  4  took  the  view  that  transformation  will  only  happen  when  these  8  “key 
elements”  co-evolve  to  take  full  advantage  of  information  age  technologies, 
capabilities  and  opportunities.  The  key  task  for  the  Force  Transformation  group 
is  to  understand  how  we  can  apply  OA  techniques  across  the  8  “key  elements” 
and  balance  the  transformation  process. 
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•  Short-term  transformation  is  occurring.  Military  forces  are  buying 
and  using  networking  capabilities.  Operational  concepts  and 
processes  are  evolving  to  harness  information  technologies. 

♦  Based  on: 

Bottom-up  and  top-down  initiatives  (iargeiy  non-program  of 
record). 

CD&E  efforts. 

Not  necessariiy  informed  by  an  “NCO  assessment”  from  the 
analysis  community. 

•  Examples:  OIF,  OEF 

•  OA  practice 

♦  Engaged  in  lessons  learned  teams,  largely  using  traditional  OA 
methods. 

♦  OA  is  not  a  driving  force. 


The  operations  analysis  community  is  marginally  engaged  in  current  short-term 
transformation  efforts.  Short  term  transformation  is  occurring  through 
operational  units  buying  or  using  information  technology  systems  and  through 
the  prototyping  efforts  of  the  Concept,  development  and  Experimentation 
(CD&E)  program. 

•  Analysts  are  engaged  in  the  “lessons  learned”  teams  deployed  to  OEF 
and  OEF  and  this  is  encouraging. 

•  Analysts  are  not  involved  early  enough  in  the  CD&E  process. 

Frequently,  the  analyst  is  not  called  upon  until  a  concept  is  ready  to  enter 
the  experimentation  phase  of  development. 

•  We  don’t  really  have  a  template  for  assessing  NCO  impacts  —  this  is 
evolving  and  analysts  need  help.  Feeding  back  the  lessons  of  recent 
operations  into  the  design  of  standing  collection  plans  and  analysis 
techniques  for  future  operations  would  create  an  environment  of 
continuous  learning. 
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of  the  Practice 

•  Long-term  transformation  initiatives  ongoing.  Top  ievel  guidance 
is  on  the  street  (NMS,  DPG,  TPG)  but  it  is  somewhat  confusing  and 
contradictory. 

♦  Translation  into  an  innovative  transformation  program  is 
problematic. 

•  Examples 

♦  Service  Efforts  (FORCEnet,  FCS)  are  not  yet  linked. 

♦  The  current  programming  and  budgeting  system  does  not  foster 
transformation. 

•  Not  Informed  by  “NCO  assessment”  from  analysis  community 

♦  No  template/guidance  for  NCO  assessment 

♦  We  are  still  early  in  the  process,  there  is  pressing  need/great 
opportunity  for  community  to  get  involved 

♦  Challenge  is  to  integrate  across  the  eight  “key  elements" 

Not  all  of  the  tools  are  “OA"  tools 


There  is  a  plethora  of  (sometimes  conflicting)  guidance  governing  US  DoD 
transformation  efforts.  Some  of  this  guidance  is  informed  by  analysis  (efforts  of 
ODS  NII/CCRP  come  to  mind)  but  additional  analysis  will  be  critical. 

Conflicting  guidance  makes  it  tough  to  develop  an  innovative,  effective 
transformation  program.  The  transformation  of  how  the  US  DoD  does  business  is 
ongoing  and  will  provide  the  foundation  and  governance  for  realizing  NCO.  This 
will  be  a  precursor  to  any  significant  long-term  deliberate  force  transformation 
being  achieved. 

Service  efforts  are  not  linked.  While  they  are  not  being  conducted  in  isolation, 
greater  collaboration  is  required. 

The  current  budgeting  feedback  system  (JROC,  DOTMLPF)  does  not  foster 
transformation.  Some  will  argue  that  this  system  is  actively  hostile  to  the  notion 
of  trading  off  platform-centric  programs  for  network-centric  programs. 

Again,  the  analysis  community  has  not  been  proactively  engaged  in  the 
transformation  process.  We  do  have  some  analytically-derived  principles 
(primarily  from  the  CCRP  body  of  work),  but  we  need  to  actively  translate  these 
principles  into  sound  analysis  practice. 

How  do  we  integrate  across  8  disparate  elements?  This  requires  us  to 
fundamentally  examine  how  we  are  currently  conducting  analysis.  The  sense  of 
our  group  is  that  the  analysis  community  is  a  bit  on  the  sidelines  in  the  current 
debate  (rhetorical  argument  has  much  more  influence  than  analysis). 
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•  Key  Issues 

♦  Current  joint  transformation  efforts  focused  on  Concept  Deveiopment 
and  Experimentation;  analysts  frequently  not  involved  until 
experiment  planning  effort  is  weli  underway. 

•  Key  Shortfalls/Gaps 

♦  Analysts  need  to  engage  early  in  transformation  initiatives  to  structure 
the  research  issues,  help  define  the  question,  and  assess  the  vaiue 
added  of  experimentation  efforts.  Applying  the  OR  method  and 
current  toois  adds  great  value. 

♦  Involvement  of  the  decision  maker  in  the  OA  process  to  create 
ownership  of  analysis.  Starts  with  formulation  of  the  question  by  the 
sponsor  with  analyst  support. 

♦  Current  experiment  program  requires  additional  analytical  rigor. 

Series  of  stand-alone  events;  no  coordinated  effort  to  develop 
experimentation  program  with  overarching  analytical  goals  and 
objectives  focused  on  delivering  enhanced  capabilities  in  the  field. 


A  specific  example  of  the  analyst  being  on  the  sideline  is  in  the  joint 
experimentation  community  where  the  analyst  is  not  brought  in  until  relatively 
late  in  the  experiment  design  phase.  This  often  can  lead  to  an  experiment  design 
that  does  not  support  the  questions  of  interest. 

This  type  of  behavior  is  symptomatic  of  the  broader  problem  of  project  sponsors 
not  being  engaged  with  the  analysis  community  in  some  circumstances.  The 
result  can  be  a  lack  of  ownership  of  the  results  of  the  analysis  performed  in 
support  of  the  study. 

The  next  step  to  integrating  analysts  into  Concept  Development  and 
Experimentation  (CD&E)  is  the  design  of  experiment  campaigns.  The  analyst 
can  assist  in  the  design  of  objectives,  data  collection  and  analysis  techniques  that 
assist  in  the  analysis  of  individual  events  as  well  as  longitudinal  analysis  between 
events. 
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•Key  Issues 

♦  As  we  move  away  from  single  service,  tactical-materiai  focused 
efforts,  traditional  OA  tools  (as  currently  employed)  are  less 
relevant  to  the  NCW  problem  set. 

•Key  Shortfalls/Gaps 

♦  Current  OA  methods  do  not  lend  themselves  to  the  study  of 
complex,  adaptive  systems  that  cross  the  four  key  NCW  domains 
(these  problems  are  at  the  core  of  defense  transformation).  Legacy 
simulation  tools  not  helpful  here. 

♦  No  coordinated  effort  to  integrate  across/co-evolve  the  eight  key 
elements  of  transformation  (concepts,  leadership,  education, 
culture,  technology,  organization,  process,  training). 


As  the  analytical  focus  of  NCW  moves  from  a  tactical  level  to  an  operational  and 
strategic  level,  the  nature  of  the  system  components  and  interactions  under 
investigation  becomes  more  complex  because  of  the  increased  focus  on 
interactions  between  humans  instead  of  technology.  In  general  there  is  a  shift 
from  quantitative  objective  measurements  to  qualitative  subjective 
measurements.  Traditional  OA  tools  do  not  support  the  upper  end  of  the 
spectrum  well. 

The  representation  of  complex  adaptive  systems  across  the  four  domains  of  NCO 
and  the  eight  elements  of  transformation  can  be  supported  by  the  softer  system 
techniques.  Techniques  that  support  strategic  planning- have  utility  in  structuring 
thoughts  and  ideas  that  can  create  a  degree  of  accountability  and  traceability  in 
these  types  of  systems. 

OA  should  look  to  other  disciplines  that  have  developed  techniques  for  dealing 
with  the  complexity  of  systems  centered  around  the  human. 
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•Key  issues 

♦  What  is  the  right  balance  between: 

Platform-centric  investments  and  information/network-centric  Investments? 

Long-term  arid  near-term  transformation  investments? 

Top-down  and  bottom-up  approaches  to  transformation?  Power  to  the  Edge) 

Investments  among  the  8  key  elements? 

Where  the  analysis  Is  done  (inside  beltway,  in  the  lab,  in  the  field)? 

♦  Which  network-centric  investments  and  initiatives  can  put  us  on  the 
path  towards  rapid,  “revolutionary”  transformation? 

•Key  Shortfalls/Gaps 

♦  Applying  OA  methods  to  quantify  benefits  of  network-centric 
investments. 

♦  What  is  the  “common  currency”  to  relate  platform-centric  and 
network-centric  investments? 

♦  Providing  relevant,  analytically-grounded  advice  to  the  “intuitive" 
decision  maker. 


Trade-off  studies  between  material  solutions  has  traditionally  been  a  rich  area  for 
OA  techniques.  With  the  need  to  trade-off  between  platforms,  networks  and 
information  a  set  of  common  metrics  are  difficult  to  develop  as  a  basis  for 
comparison.  Currently  studies  can  provide  at  best  an  indicative  assessment  of 
risk  associated  with  each  option  but  not  definitive  options  for  decision  makes. 
Thus  there  is  still  a  large  component  of  “intuition”  in  the  investment  decisions 
being  made. 
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•  Future  Challenges 

♦  Force  transfoimation  is  being  driven  by  NCO.  What  will  be  the  next 
driver  (technology  or  social  change)  for  transformation?  How  can 
OA  help  to  identi^evaluate/deploy  candidate  concepts  to  get 
ahead  of  the  game? 

♦  EBO  includes  other  government  departments,  non-government 
organizations  and  coalition  partners.  We  are  struggling  to  support 
internal  DoD  efforts  with  OA.  How  can  we  support  the  “expanded” 
network? 

♦  How  do  we  assess  entrepreneurial  initiatives  (e.g.  informal 
importation  of  COTS  systems  on  the  battiefield)? 

♦  Expanding  the  analyst’s  tool  kit,  continuously  adapting  the  OA 
process  to  evolving  problems. 


WG  4  -  Transformation 


What  is  the  next  transformational  capability  or  technology  on  the  horizon? 
Analysts  can  and  should  be  engaged  in  this  effort,  and  clearly  must  look  beyond 
the  bounds  of  defense  industries  for  promising  opportunities.  By  taking  a  broad 
look  at  the  8  “key  elements”  of  transformation  (and  beyond)  we  may  uncover 
areas  for  further  investigation.  This  is  not  a  one  player  game  -  we  must  also 
think  about  areas  that  potential  adversaries  can  exploit  and  work  them  in  to  our 
future  scenarios  and  operating  concepts. 

NCO  most  certainly  is  about  fostering  collaboration  and  networking  to  a  wide 
degree.  We  must  expand  our  network  of  analysts  to  include  interagency, 
coalition  and  non-governmental  partners. 

Use  of  PDAs  on  the  battlefield  is  an  example  of  bottom-up  approaches  that  have 
been  attempted  by  tactical  and  operational  forces.  These  typically  are  not 
officially  sanctioned.  Some  may  have  great  value,  some  may  have  minimal 
impact,  some  may  be  downright  dangerous.  Analysts  need  to  engage  in  the 
effort  to  assess  these  “bottom-up”  initiatives. 

In  a  complex,  network  centric  world  the  analysis  challenges  will  continue  to 
evolve.  We  should  not  adopt  a  mindset  of  getting  the  tools  and  processes  “right” 
-  we  need  an  ethos  of  continuously  examining  our  tools  and  processes  to  make 
them  better. 
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•  international  Community  Collaboration 


♦  Formal,  robust  programs  do  indeed  exist 

♦  Examples 

-  TTCP  (Australia,  Canada,  New  Zealand,  United  Kingdom,  United 
States) 

-  Multi-national  Interoperability  Council/Working  Groups 
(Multinational  experiment  series) 

-  NATO  initiatives 


-  ABCA 

-  COCOMTSCPs 


♦  All  are  eager  to  engage  in  these  issues. 


A  spectrum  of  change  of  military  forces  in  the  international  community  exists 
from  status  quo  to  modernization  through  transformation  of  forces.  Nations  are  at 
different  points  in  the  spectrum  with  respect  to  NCO  but  they  are  committed  to 
moving  away  from  the  status  quo  end  of  the  spectrum.  The  participation  to 
various  multinational  forums  is  evidence  of  the  commitment. 
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•Other  Considerations 

♦  Improve  balance  of  investment  advice  to  planners  and  managers  of 
transformational  programs  (strategic  planning  techniques,  cost 
benefit  analysis). 

♦  Can  analysis  help  “incentivize”  move  away  from  platform-centric  to 
network-centric  procurement  approach? 

Articulate  war  fighting  power  of  information  and  collaboration 
^  Full  range  of  capabilities 

Analysis  techniques  to  support  non-material  solutions 

♦  Has  the  OA  process  become  too  rigid?  (Unable  to  adapt  to 
complex  network-centric  problems,  unresponsive  to  decision  maker 
needs.) 


How  should  we  direct  our  transformational  efforts;  how  do  you  decide  how  to 
allocate  yom  experimentation  resources?  Operations  analysts  can  and  should 
apply  decision  analysis  techniques  and  other  methods  to  help  illuminate  these 
questions.  Studies  and  research  are  also  key  here. 

How  do  you  explain  the  war  fighting  value  of  network  centric  approaches  to  the 
military  services?  Until  you  do,  service  leadership  will  be  understandably 
reluctant  to  cancel  platform-centric  programs  in  favor  of  network-centric  forces. 
The  analysis  community  must  continue  to  develop  methods  to  articulate  the  war 
fighting  value  of  information  technologies  and  robustly  networked,  collaborative 
teaming. 

Why  was  it  necessary  to  have  a  workshop  dedicated  to  OA  support  for  Network 
Centric  (Dperations?  We  have  been  presented  with  a  problem  that  was  not  easily 
understood  using  traditional  OA  methods  and  we  needed  to  bring  the  community 
together  to  discuss  the  state  of  the  art  and  ways  to  adopt  OA  practices  to  this 
challenging  set  of  problems.  OA,  of  course,  is  fundamentally  a  flexible, 
multidimensional  approach  to  problem  solving. 
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•  Recommendations 

♦  Expand  tool  kit  and  develop  more  flexible  processes  to  examine 
robustly  networked  problems  and  complex  adaptive  systems. 

Develop  new  tools  and  techniques. 

^  Refine  existing  tools  (better,  faster,  cheaper)  to  meet 
compressed  decision  cycle. 

Less  reliance  on  formal  events,  more  effort  on  real  world  data 
collection  and  exploitation. 

Reinvigorate  the  use  of  war  gaming  for  analysis  (only  tool  that 
can  represent  the  complex  adaptive  system,  need  a  thinking, 
adaptive  adversary  that  is  allowed  to  win!).  Make  formal  events 
better! 


The  state  of  the  art  in  OA  practice  is  a  collection  of  methods  that  are  used  either 
as  a  stand  alone  application  or  combined  to  examine  multiple  aspects  of  the 
problem.  Bringing  in  techniques  from  disciplines  outside  OA  and  the  analyst’s 
tool  kit  can  become  extensive.  However  the  frameworks  for  integrating  these 
tools  and  their  results  are  an  area  for  development  of  the  OA  community  through 
meta-analysis  i.e.  analysis  of  the  OA  process.  The  guiding  principles  for  the 
application  of  the  “cross-disciplinary  tool  kit”  is  that  it  must  be  able  to  provide 
timely  results  with  an  economy  of  effort.  This  is  particularly  important  when 
dealing  with  real  world  situations  to  meet  compressed  decision  cycles. 

A  technique  that  offers  potential  to  act  as  a  framework  for  integrating  analysis 
techniques  is  a  war  gaming  construct.  The  key  elements  of  a  war  game  are  a 
realistic  representation  of  the  environment,  credible  friendly  capabilities  and  a 
thinking  and  adaptive  adversary.  The  practice  of  war  gaming  should  be 
reinvigorated  and  the  development  of  techniques  to  represent  and  measure  the 
impact  of  NCO  initiatives  need  to  be  developed. 

A  MORS  Symposium  working  group  or  composite  group  dedicated  to  NCO  is 
needed.  NCO  is  much  more  than  C4ISR  and  Information  Operations  (10).  This 
new  working  group/composite  group  could  be  charged  to  examine  the  OA  toolkit 
and  processes  in  order  to  better  focus  on  the  NCO  problem. 
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•  Recommendations 

♦  Expand  analysis  community  beyond  the  traditional  core 
competencies 

Transformation  has  8  key  elements  that  are  not  all  supported 
by  traditional  OA  competencies. 

Develop  integrated  cross  disciplinary  analysis  teams  to 
examine  cultural,  leadership,  training,  information  science 
issues. 


We  have  identified  8  elements  that  are  key  to  completing  the  transformation  to  a 
network  centric  force  (concepts,  leadership,  education,  culture,  technology, 
organization,  process,  training)  and  there  are  undoubtedly  more.  We  need  to 
expand  the  analysis  community  beyond  the  traditional  competencies  and  include 
disciplines  with  expertise  in  the  8  (or  more)  key  elements.  OA  began  as/remains 
a  multidisciplinary  approach  to  problem  solving.  We  must  continue  to  utilize 
cross-disciplinary  analysis  teams  to  address  complex  NCO  issues.  As  we  begin 
to  make  headway  on  NCO  analysis  techniques  we  must  share  new  problem 
solving  approaches  and  bring  additional  disciplines  into  the  OA  “mainstream.” 
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•  Recommendations 

♦  Use  NCO  principles  to  organize  and  employ  OA  -  Networking  and 
Collaboration 

Employ  tools  and  empower  analysts  to  increase  information 
sharing  across  discipline  boundaries  to  conduct  analysis  at  the 
“Edge.” 

♦  Lead  development  of  an  OA  “methodology”  for  network-centric 
analysis. 

^  NCW  conceptual  framework  provided  in  Power  to  the  Edge  is  a 
good  starting  point. 

What  is  the  war  fighting  value  of  information? 

Template  for  NCO  “assessments.” 


NCO  principles  need  to  be  employed  by  analysts  to  reach  out  across 
organizational  and  geographic  boundaries  to  the  skills  required  for  a  particular 
application.  Analysts  should  be  supported  by  synchronous  and  asynchronous 
collaboration  to  build  analysis  concepts,  conduct  live  and  virtual  experiments 
and  share  results. 

Many  of  the  technologies  made  available  to  industry  and  to  the  collaborative  war 
fighter  could  be  easily  extended  to  the  analysis  community. 

An  online  “community  of  practice”  website  would  go  a  long  way  to  furthering 
this  effort. 

Power  to  the  Edge  provides  a  wonderful  starting  point  -  the  “conceptual 
framework”  development  chapter  is  invaluable.  The  next  step  is  to  develop  an 
“analytical  framework”  to  guide  analysts  working  in  the  NCO  domain. 

We  still  have  some  fundamental  questions  to  address.  What  is  the  war  fighting 
value  of  information?  Is  there  a  candidate  list  of  metrics  for  conduction  NCO 
analysis?  What  are  the  fundamental  characteristics  of  the  NCO  problem? 

MORS  is  in  a  good  position  to  foster  the  development  of  an  OA  methodology  for 
NCO.  Again,  a  MORS  Symposium  working  group  or  composite  group  on  NCO 
would  be  most  helpful. 
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•  Presentations  given  in  WG  4 

♦  "Strategic  Planning  and  Force  Transformation”,  Duncan  Tailby, 
Defense  Science  and  Technology  Organization,  Australia 

♦  “Joint  Prototype  Testing",  Joe  Drelling,  Joint  Forces  Command 
J9/MYMIC  LLC 
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Operations  Analysis  Support  to 
Network  Centric  Operations 


Military  Operations  Research 
Society  (MORS)  Workshop 


WG  5:  NCO  and  Coalition  Forces 
Outbrief 


On  27-29  January  2004  WG5  met  as  part  of  the  MORS  Workshop  on 
Operational  Analysis  (OA)  Support  to  Network  Centric  Operations  (NCO). 

There  were  15  participants  in  the  working  group  from  the  US,  UK,  Canada, 
Australia  and  Sweden.  Eleven  papers  were  presented  during  the  working  group 
sessions.  There  were  three  major  objectives  of  the  working  group:  (1)  To  review 
and  discuss  papers  that  addressed  current  practice  in  the  areas  of  tools,  studies 
and  experimentation/live  events.  (2)  To  discuss  each  individual  country’s  state 
of  practice  of  OA  in  support  of  NCO,  to  identify  good  and  bad  points  and  to 
identify  future  directions.  (3)  To  assess  ‘collaborative’  efforts’  among  coalition 
partners  to  determine  shortcomings  and  to  recommend  ways  to  improve 
collaborative  OA  efforts.  Most  participants  of  the  working  group  were  experts 
and  practitioners  in  the  field  of  OA  support  to  NCO  in  their  respective  countries. 
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♦  “Insights  from  UK  Data  Link  and  Situational  Awareness  Studies”  (UK) 

♦  “NetCentric  Operations  in  the  Canadian  Contexf  (CA) 

♦  “Network  Analysis  from  a  Swedish  OR  Perspective”  (SW) 

♦  “Network  Centric  Operations  and  the  Joint  Interagency  Coordination 
Group  Workshop  Results”  (US) 

♦  "Analysis  of  Carrier  Task  Force  Network  Centric  Warfare  Architectures 
for  Network  Centric  Defense"  (US) 

♦  “An  Analytic  Approach  to  Network  Centric  Warfare  Metrics  and  their 
Relationships:  From  Connectivity  to  Operational  Tempo”  (AU) 

♦  “A  Functional  Survey  of  Netted  Warfare  Analysis  Methods  and  Selected 
Applications  to  Coalition  Maritime  Tactical  Situations”  (US) 

♦  “Dutch  and  German  Forces  Employment  of  Network-Centric 
Capabilities  During  Peace  Keeping  Operations”  (US) 

♦  “ACIDC  Developments  From  Operation  Iraqi  Freedom  Lessons 
Identified”  (UK) 

♦  “Analysis  From  Multi-National  Limited  Objective  Experiment  2”  (US) 

♦  “Analysis  Plan  for  Multi-National  Experiment  3”  (US) 


Eleven  papers  were  presented  during  the  working  group  sessions.  The  general 
order  in  which  they  were  presented  was:  (1)  Country  study  overviews;  (2)  Tools; 
and,  (3)  Experiments  and  live  events.  Papers  also  included  a  focus  on  ground 
force  NCO,  maritime  NCO  and  Air  Force  NCO.  The  papers  were  important  in 
providing  a  basis  for  discussion  to  determine  the  state  of  OA  support  to  NCO  in 
each  country.  They  were  also  important  in  order  to  understand  innovative 
methods  and  applications  in  the  participating  countries.  Two  papers  were 
presented  on  recently  completed  case  studies  of  international  NCO  in  a  NATO 
exercise  from  a  few  years  ago  and  the  use  of  “Blue  Force  tracking”  of  ground 
forces  by  the  UK  and  US  in  Operation  Iraqi  Freedom. 
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United  Kingdom 

•  State  of  the  Practice  (How  is  the  United  Kingdom  using  OA?) 

♦  Broad  toolset 

♦  Use  of  soft  OA 

♦  OA  tools  selected  based  upon  re-use  and  broad  application 

♦  Picture  models 

•  Good  and  Bad 

♦  G  -  Situational  Awareness  analysis 

♦  B  -  Linkages  between  lower  and  higher  level  models 

♦  B  -  Impact  of  not  representing  non-Integrated  Communications 
Information  System  in  modeling  (too  many  stove-pipes) 

•  Future  Challenges 

♦  Moving  into  the  cognitive  and  the  information  quality  domains  (not 
just  timeliness/comms) 

♦  Modeling  “adaptive  behavior” 

♦  High  level  investment  decisions  (NCO  vs  weapons  balance) 


The  UK  believes  that  a  networked  force  will  allow  improved  situational  awareness,  agility  and 
synchronization.  Although  a  range  of  Concepts  has  been  developed  for  NEC  (covering  areas  such 
as  the  development  of  a  Joint  Operational  Picture,  collaborative  planning,  mission  organized 
force  elements,  effects  based  planning  and  enhanced  tempo)  the  main  focus  of  NEC  investment  is 
currently  in  the  arena  of  experimentation.  The  experimentation  takes  various  forms,  through 
environments  and  organizations  such  as  NITEworks,  CBMARTD,  ACIDC,  JUEP. 

The  OA  community  has  a  wide  breadth  of  tools  available  to  imdertake  analysis  in  support  of 
NEC  activities.  These  cover  the  spectrum  from  discrete  event  simulation,  classical  OA  tools, 
sampling  and  surveys  through  to  multi  dimensional  criteria  analysis,  picture  models  and  soft 
systems  methodologies. 

The  UK  is  confident  with  the  tools  for  assessing  issues  such  as  picture  quality  and  timeline 
comparisons  at  the  tactical  level,  and  are  moving  towards  analysis  of  some  of  the  cognitive  issues 
associated  with  situational  awareness.  There  are  however  still  significant  gaps  in  the  assessment 
of  the  quality  of  information  products  at  the  higher  level.  There  are  also  limitations  in 
aggregating  information  and  knowledge  factors  between  low  level  models  and  high  level  models 
and  hence  difficulties  in  the  calibration  of  tools  at  the  Campaign  level.  Currently  analysis  often 
fails  to  represent  the  baseline  case,  (for  example  not  accounting  for  the  many  stove-piped  stand¬ 
alone  systems  that  currently  contain  the  data  used  by  decision  makers)  and  the  resultant 
limitations  in  showing  the  true  worth  of  integrated  solutions. 

The  future  challenges  are  to  increase  the  availability  of  tools  and  metrics  for  addressing  issues 
such  as  quantification  of  information  quality  and  to  develop  tools  within  the  cognitive  domain  to 
handle  adaptive  behaviour  and/or  emergent  properties.  A  key  problem  which  must  be  addressed 
is  how  to  deal  with  Balance  of  Investment  issues  above  the  immediate  NEC  level,  where  nations 
have  to  trade  networked  enabled  capabilities  at  the  expense  of  platforms  or  other  hardware, 
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Canada 

•  state  of  the  Practice  (How  is  Canada  using  OA?) 

♦  Experiments  lead,  making  use  of  some  OA  tools. 

♦  Expansion  of  activities  contingent  upon  departmental  buy-in  of 
concept 

•  Good  and  Bad 

♦  G  -  Live  experiment  identified  communication  problems/icing 

♦  G  -  Planned  Experiment 

♦  B  -  Pace  of  experimentation  effecting  ability  to  perform  OA. 

•  Future  Challenges 

♦  Challenge  of  coalitions  for  NCO  (i.e.  language,  culture,  policy, 
technology,  trust,  TTP) 

♦  NCO  applicability  across  spectrum  of  operations 

♦  Many  believe  they  are  “net-centric”  unless  shown  otherwise. 

♦  Human  factors  and  the  cognitive  domain. 

♦  Increasing  focus  on  “trust”  issues 


While  there  have  been  some  NCO-related  initiatives  within  Canada,  such  as  the 
development  of  an  Integrated  ISR  Architecture,  currently  there  is  no  national 
decision  regarding  the  adoption  of  NCO.  A  high-level  departmental  symposium 
scheduled  for  Fall  2004  is  expected  to  resolve  this  and,  hopefully,  result  in  the 
development  of  a  roadmap  for  the  way  ahead.  Meanwhile,  a  series  of  UAV 
experiments  at  the  Canadian  Forces  Experimentation  Centre,  using  some  OA 
tools,  are  working  towards  proving  the  NCO  case. 

The  use  of  planned  experiments  in  examining  the  merits  of  NCO  has  proven 
useful.  For  instance,  difficulties  with  communications,  TTP  and  UAV  icing 
were  discovered  through  experimentation  which  may  not  have  been  discovered 
otherwise.  However,  the  pace  of  experimentation  has  provided  challenges  for 
the  completion  of  OA  before  resources  are  committed  to  the  next  in  the  series  of 
UAV  experiments. 

Future  challenges  remain  for  Canada  in  the  area  of  NCO.  For  instance,  in  view 
of  Canada’s  participation  in  international  coalitions,  questions  remain  about  the 
impact  of  such  considerations  as  language,  culture,  policy  differences, 
technology  and  TTP.  Also,  the  role  of  human  factors  and  the  cognitive  domain 
for  NCO  need  to  be  explored.  For  instance,  how  is  trust  transferred  across  an 
NCO  environment  between  coalition  partners?  Moreover,  questions  exist  about 
the  use  of  NCO  across  the  full  spectrum  of  violence  (i.e.  during  low  intensity 
conflict,  peace  support  operations,  etc.).  Importantly,  especially  in  view  of 
Canada’s  participation  in  coalition  warfare,  the  issue  of  how  trust  is  transmitted 
across  an  NCO  environment  between  coalition  partners  needs  to  be  explored. 
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Sweden 

•  State  of  the  Practice  (How  is  Sweden  using  OA?) 

♦  OA  support  to  national  NCO  experimentation 

♦  Thought  pieces/analyzing  relevant  literature  and  simple  tools 

♦  Trade-off  between  analyzing  Investment  decisions,  networking  soldier  vs  system-of- 
systems  vs  high  level  C2  systems 

♦  Found  it  useful  to  use  social  analysis  network  tools 

•  Good  and  Bad 

♦  G  -  View  networks  as  enablers/threats/targets/tools 

♦  G  -  Assessed  cost  of  developing  information  infrastructure 

♦  B  -  Previous  analysis  made  unreasonable  assumptions  about  network  capabilities 

♦  B  -  Standard  methods  of  Network  Reliability  unsuited  for  highly  dynamic,  mobile 
networks 

♦  B  -  Lack  of  suitable  simulation  models  for  operational  effectiveness  Impacts  of  NCO 

•  Future  Challenges 

♦  Assessing  interoperability  requirements  in  NCO  Qolnt,  combined  and  interagency) 

♦  Analyzing  complex  networks 

♦  Making  sure  “does  model  match  reality?” 

♦  OA  on  how  to  attack  and  defend  a  network 


The  current  main  NCO-related  OA  effort  at  the  Swedish  Defence  Research  Agency  (FOI)  is  our  support  to 
the  Swedish  Network-based  Defence  experimentation  program,  where  FOI  is  tasked  to  evaluate  the  ongoing 
system  demonstrator  program. 

FOI  also  supports  the  long-term  planning  process  within  the  Swedish  Armed  Forces  Headquarters,  providing 
analysis  in  support  of  the  Long-term  Parliamentary  Defence  Decision  2004.  FOI  was  tasked  with  developing 
and  providing  coarse  cost  estimates  of  alternative  network  structures,  coupled  to  different  future  force 
structures.  The  main  emphasis  has  been  on  the  network  components  necessary  to  ensure  a  joint  networking 
capability.  Early  in  that  work  we  pointed  to  the  basic  balance  of  investment  between  networking  soldiers, 
platforms  and  C2  systems,  respectively. 

Given  limited  resources  and  a  general  lack  of  large,  suitable  simulation  models,  we  are  forced  to  focus  more 
on  soft  OA  methods,  literature  studies  and  simple  models. 

We  take  a  broad  view  of  the  role  of  networks  in  the  field  of  Defence  Analysis,  which  can  be  outlined  in  the 
following  four  aspects:  1)  Networks  as  enablers;  2)  Networks  as  Threats;  3)  Networks  as  targets;  and,  4) 
Networks  as  tools. 

The  first,  networks  as  enablers,  is  of  course  the  main  aspect,  and  encapsulates  all  our  current  efforts  to 
leverage  the  power  of  networks  in  developing  new  military  capabilities  and  concepts  of  operation. 

The  second  aspect  covers  the  idea  that  various  criminal  and  terrorist  groups  are  effectively  early  adopters  of 
networking.  If  you  believe  this,  a  better  understanding  of  network  forms  of  organization  and  related  concepts 
of  operation  is  required  not  only  to  transform  our  armed  forces,  but  also  to  understand  the  character  of  future 
conflicts  in  general. 

The  third  aspect  raises  the  obvious  fact  that  in  network-centric  operations,  your  networks  will  be  prime 
targets  for  enemy  attack.  More  broadly,  it  recognizes  that  all  our  values  at  risk  in  future  conflicts  are 
increasingly  networks  per  se\  civilian  infrastructure  is  largely  an  interconnected  set  of  co-dependent 
networks.  The  proper  way  to  identify,  evaluate  and  then  mitigate  these  new  vulnerabilities  is  however  far 
from  obvious. 

The  fourth  aspect  ties  it  all  together  and  starts  with  the  simple  question:  Do  we  as  OR  analysts  have  the 
proper  tools  and  the  necessary  understanding  to  deal  with  all  other  aspects  of  network  centric  operations  in 
the  future?  Project  Metanet  is  a  multi-year  competence-building  activity  to  incorporate  recent  findings  from 
such  diverse  areas  as  complexity  studies,  bioinformatics,  and  Social  Network  Analysis,  into  our  work  at  the 
Swedish  Defence  Research  Agency. 
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Australia 

•  State  of  the  Practice  (How  is  Australia  using  OA?) 

♦  Defining  the  concept  of  NCW  via  Concept  Development  and 
Experimentation 

♦  OA  to  support  NCW  acquisition  decisions 

♦  Develop  of  agent  based  approach  for  NCW 

♦  Developing  quantifiable  metrics  for  knowledge  f(Accuracy,  Precision,  Bias) 

•  Good  and  Bad 

♦  G  -  Understanding  of  information  sharing  from  Internal  standpoint 

♦  G « Simple  experiments  providing  insight  to  NCW 

♦  G  -  Using  system-of-systems  approaches  to  NCW 

♦  B  -  Need  more  NCW  tools 

•  Future  Challenges 

♦  Social  network  analysis  applied  to  NCW 

♦  Broaden  OA  applicability  to  multi-disciplinary  areas  to  include  cognitive 
modeling 

♦  How  to  represent  knowledge  (e.g.  effusion  of  data  Into  knowledge) 


Network  Centric  Warfare  (NCW)  is  recognised  as  a  key  element  in  maximising  the  Australian 
Defence  Force  (ADF)  military  effectiveness  both  the  Chief  of  Defence  Forces  vision  for  the 
ADF,  Force  2020  and  Future  Warfighting  Concepts,  documents.  The  latter  document  describes 
NCW  as:  “Network-centricity  will  help  us  to  link  national,  ADF  and  coalition  sensors, 
engagement  systems  and  decision  makers  into  an  effective  and  responsive  whole.  At  its  core, 
NCW  seeks  to  provide  the  future  force  with  the  ability  to  generate  tempo,  precision  and  combat 
power  through  shared  situational  awareness,  clear  procedure  and  the  information  connectivity 
needed  to  synchronize  our  actions  to  meet  the  commander’s  intent.” 

To  better  understand  and  assess  the  benefits  and  vulnerabilities  of  networked  operations,  and  to 
provide  timely  advice  to  our  senior  decision  makers  on  balance  of  investment  (e.g.  NCW  versus 
weapons),  we  need  to  carry  out  Operations  Analysis  (OA)  using  a  range  of  tools.  In  Australia,  we 
are  defining  the  concept  of  NCW  via  concept  development  and  experimentation.  We  have  found 
that  small  scale  experiments  in  the  form  of  seminar  wargames  and  matrix  games  to  be 
particularly  useful  for  gaining  insights  into  NCW.  We  are  also  developing  agent-based  techniques 
for  modeling  NCW.  The  value  of  quantifiable  metrics  for  knowledge  assessment  is  also 
recognized. 

Our  current  advantage  is  in  our  comparatively  good  understanding  of  information  exchange, 
skills  in  conducting  small-scale  experiments  and  using  systems  of  systems  approach  to  analyze 
NCW.  On  the  other  hand,  we  note  that  many  more  tools  have  yet  to  be  developed  to  assess  the 
benefits  of  NCW. 

Some  of  the  challenges  include  the  inclusion  of  social  network  analysis  techniques  in  our  NCW 
analysis  and  the  need  to  broaden  OA  capability  to  multi-disciplinary  areas,  including  social  and 
cognitive  modeling.  A  proper  representation  of  knowledge,  e.g,  via  fusion  of  data  into 
knowledge,  remains  elusive. 
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•  Recommendations 

1 .  Ensure  lessons  learned  teams  have  OA  team  members 

2.  Develop  a  process  to  collect  data  from  real  world 
operations/experiments/training  and  feed  back  into  NCO  analysis. 

3.  Create  opportunities  for  experiments  to  explore  social  and  cognitive 
domain  issues  of  NCO. 

4.  Develop  OA  methods  and  metrics  for  assessment  of  social  and  cognitive 
domain  issues  of  NCO. 

5.  Assess  biological  and  other  adaptive  systems  network  analysis  tools  for 
applicability  to  NCO. 

6.  Advocate  the  use  of  case  study  methodoiogy  in  the  analysis  of  NCO. 

7.  Develop  quantitative  techniques  for  assessing  the  quality  of  information. 

8.  Conduct  an  assessment  of  interoperabiiity  requirements  in  NCO  (joint, 
combined  and  interagency). 

9.  Conduct  an  increased  emphasis  on  NCO  modei  validation. 

1 0.  Conduct  analysis  on  attack  and  defense  of  networks. 


The  recommendations  were  derived  primarily  from  the  ‘Future  challenges’  identified  by 
each  individual  country  and  agreed  to  by  the  entire  group  as  applicable  to  all  coalition 
countries.  The  recommendations  included  how  to  provide  OA  support  in  experiments  or 
real  world  events  (recommendations  #l-#3).  There  was  the  recognition  among  coalition 
country  representatives,  as  in  most  of  the  other  previous  working  groups,  that  there  were 
shortcoming  in  doing  OA  in  the  social  and  cognitive  domains  when  following  the  NCO 
Conceptual  Framework  (#4).  Sweden  suggested  that  biological  and  adaptive  system 
network  tools  could  be  very  beneficial  when  trying  to  understand  how  to  build  or 
understand  robust  networks  (#5).  During  the  out  brief  the  Army  Sponsor  independently 
made  this  same  recommendation.  As  a  result  of  the  perceived  benefit  of  the  two 
international  case  studies  presented  during  the  sessions  it  was  recommended  that  this 
approach  be  adapted  as  a  viable  technique  for  doing  OA  for  NCO  (#6).  But  it  was 
suggested  that  it  was  necessary  to  have  the  same  analytic  rigor  and  objectivity  applied  to 
the  case  study  methodology  as  with  other  analytic  techniques.  During  the  sessions  there 
was  general  agreement  that  current  OA  tools  are  adequate  to  assess  network  speed  of 
service  and  latency.  However,  it  was  pointed  out  that  OA  tools  are  not  adequate  to  assess 
the  impact  of  various  levels  of  the  quality  of  information  (#7).  It  was  agreed,  from  a 
coalition  point  of  view,  that  a  major  purpose  of  analysis  is  to  assess  interoperability 
between  coalition  partners.  A  major  shortcoming  is  that  there  has  never  been  an 
assessment  of  the  requirements  for  interoperability  between  various  coalition  partners 
which  would  lead  to  operational  architectures  and  system  architectures  (#8).  Coalition 
countries  agreed  that  it  is  critical  when  developing  all  NCO  models  that  we  should  never 
lose  sight  of  the  stringent  requirement  that  all  NCO  models  be  subject  to  rigorous 
assessments  to  insure  that  models  match  reality  (#9).  Finally  given  the  view  that  all 
coalition  forces  will  be  heavily  reliant  on  networks  in  any  future  military  operation  it  is 
important  that  significant  analysis  be  undertaken  to  determine  the  impact  of  network 
attacks  and  to  develop  means  to  protect  networks  (#10). 
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A  quick  survey  was  undertaken  during  the  working  group  session  to  identify  the 
techniques  coalition  country  representatives  felt  were  most  applicable  to  NCO. 
Each  country  was  given  three  “votes”  that  they  could  apportion  among  the 
various  tools.  The  guidance  was  that  they  should  identify  the  tools  or  techniques 
that  were  most  relevant  and  useful  now  and  in  the  immediate  future  for  OA 
support  to  NCO.  It  is  interesting  that  there  was  not  overlap  by  any  three 
countries  for  a  particular  technique  but  there  was  double  overlap  for  the 
techniques  of  military  war  gaming,  human-in-the-loop  computer  modeling, 
discrete  event  simulation  (normally  large  force  effectiveness  models  that  are 
where  networks  are  overlaid  on  platform  centric  models),  agent  based  simulation 
(countries  are  staring  to  use  simulations  such  as  the  ‘Mana’  simulation),  and 
experimentation.  Representatives  felt  that  experiments  should  be  focused  and 
include  clear  hypothesis  that  can  be  confirmed  or  denied.  They  should  not  be 
demonstrations  without  rigorous  experimental  design  and  there  should  be 
analysis  plans  for  data  collection. 
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•  OA  Collaborative  Efforts  (between  coalition  partners) 

♦  UK-US-AU  knowledge  representation  using  entropy  work 

♦  OFT  2  international  efforts,  US-UK,  NATO 

♦  US-UK-AU-CA-NZ  -  TTCP  -  agent  based  modeling  and  queuing 
theory  (maritime  NCO) 

♦  US-UK-AU-CA-NZ  -  TTCP  looking  at  S&T  implications  of  NCO 

♦  MIC  Nations  -  NATO  (J9/APL)  Decision  making  in  collaborative 
environment  (human  factors  analysis) 

♦  ESDAG  -  UK-GE-SWE-NL  -  Interoperability  analysis  related  to  NCO 

♦  US-UK  interoperability  study  -  UK  BDE  in  US  div  in  NCO 
environment 

♦  NATO  SAS50  study  -  development  of  a  conceptual  model  for  C2 
assessment 

♦  ABCA  C4I  Exercise  -  Working  group  to  discuss  interoperability  and 
C4I  focus 

♦  J9  focus  on  making  all  experimentation  events  multinational 


The  third  major  objective  of  the  working  group  was  to  assess  the  state  of 
coalition  ‘collaborative’  OA  analyses  in  support  of  NCO.  This  began  by 
identifying  all  known  recent  or  planned  analytical  events.  Many  of  the  activities 
are  modest  events  between  coalition  partners  (entropy  work,  agent  based 
modeling,  S&T  assessment,  an  assessment  of  decision  making  in  collaborative 
environments,  development  of  a  conceptual  model,  and  working  groups  to 
discuss  interoperability).  The  two  ‘Office  of  Force  Transformation’  (OFT)  case 
studies  are  also  modest  level  efforts  primarily  being  done  by  the  US  to  assess 
coalition  interoperability.  The  only  larger  efforts  are  the  US-UK  interoperability 
study  and  the  US  Joint  Forces  Command  (JFCOM  J9)  experiments  that  include 
significant  multinational  participation  and  interoperability  assessments.  The 
conclusion  is  that  there  is  still  a  need  for  significant  collaborative  anal5dical 
events  between  coalition  partners. 
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•  OA  Collaborative  Efforts  (shortfalls) 

♦  Lack  of  accepteci  OA  tools  to  assess  NCO  within  the  social  and 
cognitive  domains 

♦  Lack  of  pre-NCO  baseline  performance  analysis 

♦  Lack  of  analysis  on  key  features  of  NCO  (important  to  the 
international  community) 

♦  NCO  is  stove-piped  (e.g.  by  service,  by  country,  by  discipline) 

♦  International  sharing  of  data 

♦  OA  analysis  timelines  are  not  fast  enough  to  meet  the  pace  of 
technology  and  DOTMLPF  development 

♦  We  may  not  have  the  correct  Measures  of  Merit  (MoMs) 

♦  Cannot  adequately  represent  all  the  benefits  of  NCO  in  combat 
models  (e.g.  self-synchronization  and  operations  tempo) 

♦  OA  is  overly  focused  on  military  to  the  exclusion  of  civilian  agencies 

♦  OA  must  be  careful  to  represent  the  balance  to  include  unintended 
consequences  and  characterizing  risks  and  benefits 

♦  Need  to  choose  OA  events  with  mutual  benefits  within  the 
international  community 


Shortfalls  were  identified  based  on  the  recent  OA  collaborative  efforts.  Coalition 
country  representatives  identified  shortcomings  based  on  their  understanding  and 
participation  in  recent  collaborative  efforts. 
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•  OA  Collaborative  Efforts  (recommendations) 

1 .  Leverage  experiments  with  particular  focus  on  cognitive  and 
social  domain  issues 

2.  Assist  international  partners  with  their  NCO  analysis 

3.  Focus  on  DOTLP  (PRICIE)  not  Materiel  solutions 

4.  Leverage  NC3A  for  collaborative  NCO  research/analysis 

5.  Support  OA  studies  on  force  transformation  that  include 
interagency  interactions 

6.  Conduct  more  OA  focused  NCO  events 

7.  Wide  dissemination  of  international  studies  and  OA  lessons 
learned 


There  were  seven  recommendations  to  improve  ‘collaborative’  OA  in  support  of 
NCO  between  coalition  partners.  The  working  group  determined  that  it  was 
important  to  focus  on  cognitive  and  social  issues  when  doing  collaborative  OA 
assessments  (recommendation  #1).  The  working  group  determined  that  when 
conducting  collaborative  analysis  there  is  a  requirement  to  represent  key 
networked  entities.  However,  there  were  cases  where  ‘disadvantaged’ 
participants  may  not  be  represented  if  there  is  not  assistance.  A  recommendation 
is  to  provide  assistance  to  insure  that  key  disadvantaged  players  are  part  of 
important  analysis  (#2  and  #5).  There  have  been  some  analytical  events  (e.g.  J6 
Sensor  to  Shooter  US-UK  study,  US-UK  FSCS/Tracer  study)  that  have  resulted 
in  recommendations  for  investments  by  coalition  partners.  There  is  evidence  that 
there  is  a  reluctance  to  provide  necessary  resources  to  support  ‘materiel’ 
investment  recommendations.  Therefore  a  recommendation  is  that  countries 
focus  on  non-materiel  alternatives  in  their  analysis  first  (doctrine,  organization, 
training,  logistics,  personnel  —  #3).  A  concern  was  that  NATO  has  a  relatively 
large  analytical  organization,  NCSA,  that  did  not  participate  in  this  workshop. 
Given  the  emphasis  in  OA  support  to  NCO  by  a  number  of  NATO  countries  it 
was  recommended  that  NCSA  take  an  active  role  to  sponsor  collaborative 
analysis  in  this  area  (#4).  At  the  conclusion  of  the  workshop  all  participants 
agreed  that  this  workshop  and  working  group  proved  very  useful.  There  was 
strong  agreement  that  there  should  be  more  international  OA  events.  Also,  there 
was  strong  agreement  that  all  countries  should  do  much  better  at  sharing  studies, 
lessons  learned  and  data. 
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WG  6:  Applying  NCO  to  an  Actual  Event 
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•  State  of  the  Practice 

♦  Level  of  applying  OA  to  an  NCO  event;  Just  Starting 

♦  Applications/Examples  - 

■^•CAA  Deployable  Analysis  Support  Team  (DAST)  support  to  OIF 
^CENTCOM  Time  Sensitive  Target  (TST)  Manager  process  reduced 
average  TST  time 

■»OA  Collaborative  capability,  initiated  by  the  JLLT,  enabled  quick-win 
solutions  to  OIF  challenges 

♦  NCO  Assessments  and  the  OA  Process  - 

■^OA  currently  limited  by  available  AND  valid  data 

^  Dynamic  environment;  the  data  is  not  clean  or  consistent 

-  Warfighting  systems  don’t  output  OA-ready  data 

-  Data  collection  must  be  pre-planned  due  to  the  numerous  organizations  involved  and  their 
focus  on  the  real-world  operation 

■^OA  process  (i.e.  experimentation  with  ADOCS)  enabled  NCO 
application  to  TST  process 
-^Tirne  and  data  constraints  limit  the  tool  set 

-  Data  available  essentially  limits  after-action  tools  to  surveys  (interviews)  and 
spreadsheets 

-  Unable  to  provide  timely  simulation  support  to  real-world  events 

-^Reliance  on  descriptive  analysis  prevents  ability  to  explore  sensitivity 


Examples: 

•  CAA  embedded  an  analyst  in  the  CFLCC  C5  during  OIF  and  currently 
have  one  deployed  to  CJTF-7 

•  CENTCOM  TST  Manager  was  a  collaborative  link  between  the  disparate 
organizations  in  the  TST-approval  process 

•  JLLT  analysis,  enabled  by  a  robust  collaborative  capability,  provided 
“quick-win”  (rapid  tum-around)  support  to  CENTCOM  and  the  component 
commands  during  OIF 

NCO  Assessments  and  the  OA  Process: 

•  Deployed  CAA  analysts  discovered  that  data  and  info  was  not  being 
collected  or  stored. 

•  Available  data  was  often  only  in  “text”  format  and  lacked  discipline  in  its 
collection 
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•  Key  Issues,  Shortfalls  and  Gaps 

♦  Key  Issues 

^US  is  not  adequately  postured  to  perform  OA  for  actual  events 

-  OA  provides  focus  for  NCO  among  staffs  and  Joint,  Interagency.  Multinationa) 

-  UK  may  provide  Insights  toward  improving  processes  and  procedures 

^Lack  of  joint  and  some  service  doctrine  to  incorporate  analysis  in  a  JTF 
limits  OA  of  NCO.  Reliance  on  non-embedded  analysts  makes  OA  an 
afterthought  in  JTFs. 

^OA  support  to  NCO  events  is  not  networked;  reach-back  capability  not 
exploited. 

♦  Key  Shortfalls/Gaps 

^Incomplete  data  collected  from  actual  events  due  to  ‘retro-fitting’ 
instrumentation. 

^Joint  and  Coalition  non-interoperability. 

■♦OA  support  hindered  by  not  applying  analysis  during  crisis  action 
planning,  deployment  and  employment  to  implement/improve  NCO. 

^Classification  boundaries  and  releasability  restrictions. 

■♦Application  and  data  incompatibility. 


Key  Issues: 

•  CAA  had  to  overcome  inertia  to  get  its  analysts  accepted  into  the  combatant  command.  Typical 
staff  officer  and  leaders  are  unfamiliar  with  the  tools  and  capabilities  brought  by  a  trained 
analyst. 

•  UK  analysts  were  well-embedded  into  deployed  UK  forces  and  have  a  history  of  being 
embedded  in  deployed  forces.  That  model  for  emplo5rment  of  analysts  could  be  a  start  for  the 
US  program. 

•  Deployed  OA  assets  were  not  aware  of  each  other’s  efforts.  Teams  went  into  theater,  often  on 
short  notice,  and  did  not  have  time  to  contact  other  agencies  prior  to  deployment  to  become 
familiar  with  and  coordinate  efforts.  Lack  of  collaboration  between  teams  (except  in  selected 
cases)  prevented  sharing  techniques  and  data. 

Key  Shortfalls/Gaps: 

•  Warfighting  systems  not  designed  to  capture  OA  data.  When  data  is  desired,  the 
instrumentation  to  capture  it  had  to  be  developed  and  implemented  “on  the  fly.” 

•  Warfighting  systems  are  often  non-interoperable,  so  data  being  collected  would  not  be  in  any 
kind  of  standard  format. 

-  Example:  Falconview  (USAF)  and  C2PC  (USA)  cannot  interact  without  extensive 
modification 

•  Lack  of  OA  during  crisis  action  planning,  deployment  and  employment  is  an  overall  OA 
shortfall  (not  just  for  NCO). 

•  Classification  and  releasability  problems  are  also  not  just  NCO-related. 
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•  Future  Challenges 

♦  Information  sharing  will  get  more  difficult  due  to  increasing  reliance 
on  SiPRNET  and  classifications. 

♦  Increased  analysis  requires  a  change  of  culture . 

■^Doctrine  development  and  acceptance  not  instantaneous 
^Leaders,  operators  will  require  more  analytic  skills  to  effectively  exploit 
NCO 

■^Utilization  of  OA  to  facilitate  change  in  systems,  processes  and  force 
structures 

♦  Lack  of  procedures  for  data  collection  during  conflict  planning 
processes. 


If  we  don’t  do  anything,  the  info  sharing  problem  will  actually  get  worse  because 
the  planning,  execution  and  analysis  processes  are  conducted  increasingly  on  the 
SIPRNET.  This  has  the  effect  of  excluding  our  coalition  partners. 

Overclassification  of  planning,  deployment  and  execution  and  sustainment 
products  exacerbates  the  information  sharing  problem.  This  adds  work  to  the 
declassification  work  that  the  staff  Foreign  Disclosure  Officers  must  do  prior  to 
transferring  products  to  coalition  information  sharing  systems  like  CENTRIX. 

The  warfighting  culture  needs  to  be  changed  to  accept  more  analysis.  This  will 
require  changes  in  doctrine  and  training  and  experience  of  leaders. 

As  analysts  do  more  collection  and  analysis  in  the  field,  the  procedures  for  data 
collection  will  mature,  but  emphasis  on  collecting  data  should  start  in  the 
planning  phase. 
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•  Extent  of  Collaboration 

♦  Among  the  Services 

■♦Level  -  varies 
^Examples 

-  TST  Manager:  Frequent  (between  operators) 

-  JLLT:  Frequent  (between  analysts) 

-  Deployed  service  analysts:  Infrequent  (between  analysts) 

♦  With  the  Coalition  Forces 

^  Level  -  Non-existent 

♦  With  the  US  Agencies 

^ Level -varies 
■♦Examples 

-  Departments  of  State,  Education,  Treasury,  Justice.  Commerce,  etc.:  Infrequent 

-  OGA:  Inconsistent 


Collaboration  between  services  is  varied  and  has  two  dimensions:  between 
operators  and  between  analysts.  The  TST  manager  is  an  example  of  continuous 
collaboration  between  operators.  Collaboration  between  analysts  was  good  with 
the  Joint  Lessons  Learned  Team  (JLLT)  because  they  were  located  at  all 
Combatant  and  Component  Headquarters  so  contact  was  physical.  Collaboration 
between  service  teams  at  disparate  sites  was  very  infrequent. 

No  collaboration  appears  to  have  been  done  in  OIF  between  US  and  coalition 
analysts.  Most  seemed  unaware  of  the  others’  existence. 

There  was  some  collaboration  with  OGA  analysts,  but  very  little  collaboration 
between  anyone,  much  less  analysts,  in  DOS,  DoEd,  DoCommerce,  etc.  OIF 
Phase  IV  planning  involved  almost  every  major  agency  in  the  US  government 
and  none  of  them  had  sufficient  collaborative  capabilities  and  expertise  to 
contribute  effectively  and  in  a  timely  manner. 
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•  Other  Considerations 

♦  Improve  transmission/bandwidth  issues  and  limitations 

^N/A  (No  input  on  the  bandwidth  issue.) 

♦  Minimize  fratricide,  IFF  and  Combat  ID  issues 

■^TST  Manager 

♦  Fuse  data 

•>IED  Analysis 
^CJTF7  Summary  Slides 
■^MOEs  from  Campaign  Plan 
■♦CFLCC  Term  Analysis  Cell 


The  TST  manager  was  a  good  example  of  a  NCO-enabling  system  that 
contributed  to  minimizing  fratricide.  No  other  systems  could  be  identified  in  the 
discussion. 

There  were  many  examples  of  data  fusion  some  from  Combat  Operations  (Term 
Analysis)  and  the  rest  from  the  post  Combat  phase. 

•  Term  brought  together  analysts  from  many  disciplines  to  map  the  oil 
distribution  system  (for  instance). 

•  lED  analysis  did  the  same  but  to  find  solutions  to  the  Improvised  Explosive 
Devices  (lED)  causing  problems  for  coalition  forces. 

•  CJTF7  tried  to  provide  multiple  looks  at  on-going  operations  in  order  to 
help  the  commander  gauge  progress. 
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•  Recommendations 

♦  Specific  to  WG  area 

•^Network  analysts  with  collaborative  capabilities 
JFCOM/Combatant  Commands  include  coded  billets  for  OA  analysts  in  SJFHQ 
-^Develop  methodology/tools  to  enhance  OA  support  within  time  constraints 

♦  Overall  to  OA  Support  to  NCO 

JFCOM  develop  joint  doctrine  for  OA  Support 

-  standard  for  collecting  and  archiving  data 

JFCOM  develop  products  and  processes  to  facilitate  NCO  Information  sharing. 
JFCOM/Combatant  Commands  Integrate  OA  support  across  all  phases  of 
planning  through  operations  to  enhance  NCO 

-  Include  analysts/analysis  during  JTF  training 

^Develop  capability  to  instantiate  constructive  simulations  from  real-world/current 
data 

♦  Other  comments 

■♦Coalition  information  sharing  constrains  collaboration  between  analysts  as  well  as 
operators 


•  Specific  recommendations  include: 

-  Networking  analysts  so  that  they  can  exchange  techniques  and  data 

-  Code  billets  in  the  SJFHQ  for  OA  analysts  so  that,  when  they  plug  into 
a  JTF,  analysts  focused  on  performing  the  OA  function  will  be 
available. 

-  Develop  rapid  tum-around  models  and  other  analytic  tools  to  assist 
deployed  analysts. 

•  Overall  Recommendations: 

-  There  needs  to  be  doctrine  as  well  as  tools  and  processes  in  order  to  get 
OA  support  into  the  field  to  assist  fi’om  planning  through  post 
hostilities. 

-  A  simulation  that  can  take  the  data  from  the  Common  Operating  Picture 
and  “run  forward”  to  provide  analysis  of  alternative  course  of  action 
could  have  great  utility  for  analysts  to  assist  the  commander. 

•  The  US  has  to  come  to  grips  with  the  coalition  information  sharing  issues  to 
assist  operations  and  the  analysis  of  operations. 
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•  Presentations 

♦  "Joint  Lessons  Learned:  Network  Centric  Warfare  Observations  in 
Operation  IRAQI  FREEDOM,” hy  Maj  Michele  Cook,  Joint  Lessons 
Learned  Team,  USJFCOM 

♦  "Providing  Analytical  Support  to  SUCA  Phase-IV  Ops  with 
Representation  in  Theater  (PASSPORT),”  by  CRT  Allison  Stewart, 
Center  for  Army  Analysis 
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•  Current  State  of  Health  of  the  OA  Process 

♦  Problem  Structuring:  Yellow 

♦  Measures  of  Merit:  Orange 

♦  Human  Factors/Organization:  Red 

■^No  social  sciences  aspect  of  OA  training 

♦  Scenarios:  Orange 

♦  Data:  Red 

•♦No  data  on  performance  of  networks 

♦  Tools:  Orange 

♦  Risk  and  Uncertainty:  Orange 

♦  Report:  Yellow 
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WG1 
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Mike  Bauman,  FS 
Jeff  Kline 

USA 

USN 

WG2 

Methods  and  Process 

Greg  Parnell,  FS 

USMA 

WG3 

Assessment  Tools 

Dave  Garvey 

Consulting 

WG4 

Force  Trans.  Process 

Bob  Manke 

USN 

WG5 

NCO  and  Coalition  Forces 

Gene  Visco,  FS 

Consulting 

WG6 

Applying  NCO 

Bill  Kemple 

NPS 

Floaters 

Chris  Herstrom 
Jerry  Kotchka,  FS 

Industry 

Industry 

Synthesis  Team 

The  Synthesis  Team  was  created  to  develop  a  holistic  view  of  the  deliberations 
of  all  of  the  Working  Groups  participating  in  the  Workshop.  Consistent  with 
that  goal,  a  team  was  selected  that  included  representation  from  the  Services, 
industry,  consulting  and  academia.  Selected  members  of  the  Synthesis  Team 
actively  participated  in  each  of  the  Working  Groups  (see  the  Viewgraph  for  the 
specific  assignments).  The  Synthesis  Team  convened  periodically  during  the 
course  of  the  Workshop  to  discuss  and  compare  the  evolving  findings  and 
recommendations  of  each  Working  Group.  Drawing  on  those  deliberations,  the 
Synthesis  Team  developed  some  broad  observations  about  the  nature  of  the 
problem  and  formulated  a  set  of  cross-cutting  findings  and  recommendations. 
These  products  are  presented  and  discussed  in  this  presentation. 
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•  To  provide  assessment  to  revitalize  the  state  of 

analytical  practice.  “State  of  Health” 

•  To  provide  a  roadmap  to  revitalize  the  state  of  anaivtical 

practice.  “Findings” 

•  To  recommend  priorities  of  any  initiatives  identified  (in 

the  roadmap).  “Recommendations” 

•  Areas  to  consider 

♦  Metrics 

WG1 

♦  Analytical  processes/methodologies 

WG2 

♦  Modeling  and  Simulation 

WG3 

♦  NCO  and  Force  Transformation  Process 

WG4 

♦  NCO  assessment  collaboration 

WG5 

♦  Quick  turn  analyses  of  NCO  enables  an  event 

WG6 

The  Synthesis  committee  organized  their  output  to  answer  the  overall  objectives 
of  the  Terms  of  Reference  (TOR). 
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•  Participation  of  senior  service  anaiyticai  ieaders  and 
experienced  anaiysts  from  all  service  and  joint 
commands  in  MORS  Workshops  significantly 
enhance  interactions,  findings  and 
recommendations. 

•  Network  Centric  Operations  are  inherently  joint, 
which  suggests  a  need  for  a  “joint”  business  model 
that  enables  services  to  conduct  operations  analysis 
in  support  of  “born  joint”  NCO  concepts. 

•  The  networking  of  operations  analysis  centers  and 
analysts  to  support  NCO  needs  to  be  addressed  in 
order  to  enhance  deployment,  experiments  and 
assessments. 

•  For  NCO  analyses,  a  set  of  measures  of  merit 
(MOMs),  not  a  single  MOM  needs  to  be  addressed. 


As  a  result  of  the  Synthesis  Team  discussion  of  on  going  activity,  some  broad 
observations  were  generated  and  presented  to  the  workshop. 

First,  there  exists  a  need  to  include  senior  service  analytical  leaders  and 
experienced  analysts  in  MORS  workshops  to  enhance  interactions,  findings, 
recommendations  and  to  mentor  more  jimior  participants. 

Second,  a  joint  business  model  is  needed  that  enables  services  and  joint 
commands  to  conduct  operations  analysis  in  support  of  emerging  NCO  concepts. 
This  will  facilitate  the  initiation  of  OA  activity  early  in  the  planning  process. 

Networking  of  operations  analysis  centers  and  operation  analysts  along  with 
adequate  reach  back  should  provide  a  significant  improvement  in  support  of 
decision  makers.  However,  an  environment  of  trust  is  needed  among  analysts 
and  between  the  analyst  and  decision  maker. 

The  final  broad  observation  is  that  NCO  analysis  will  not  be  addressed  by  a 
single  MOM  but  by  a  set  of  MOMs. 
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Major  Issues 
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•  Measures  of  Merit 

♦  Measuring  NCO  “effectiveness"  may  require  generation  of  non- 
traditional  MOEs  to  evaluate  cognitive  and  social  domains. 
However,  traditional  warfighting  MOEs  could  still  be  applicable 
as  we  use  NCO  to  do  the  same  missions  differently. 

•  Processes  and  Methodologies 

♦  Since  the  framework  emphasizes  “Quality  of  Individual 
Sensemaking"  and  “Quality  of  Shared  Sensemaking"  as  the 
major  processes  in  the  cognitive  domain,  the  lack  of  processes 
and  methodologies  for  OA  using  sensemaking  and  cognitive 
modeling  make  the  NCO  framework  high  risk. 


A  major  consideration  is  to  initiate  the  development  of  NCO  MOMs  as  new 
NCO  concepts  and  experiments  are  developed. 

The  NCO  framework  presented  at  the  plenary  session  was  evaluated  after 
considerable  discussion.  A  major  issue  was  the  gap  between  sensemaking  and 
decision  making,  along  with  a  lack  of  knowledge  as  to  the  “state  of  the  analysis” 
for  sensemaking  and  cognitive  modeling. 
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Major  Issues  (Continued) 
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•  Assessment  Tools 

♦  The  current  tools  need  improvement.  The  “data 
domain”  (both  real  blue  and  red  data  -  current  and 
projected)  is  still  too  stovepiped. 

•  Force  T ransformation  Process 

♦  Current  joint  efforts  focused  on  Concept 
Development  and  Experimentation  (CD&E);  analysts 
frequently  not  involved  until  well  after  problem 
definition. 

♦  For  trade  off  studies  between  platform-centric  versus 
network-centric,  the  OA  techniques  for  quantifying 
network-centric  investment  are  lacking. 


Each  working  group  assessed  that  the  data  domain  remains  an  area  for 
significant  improvement.  The  need  for  “standard,  approved”  data  for  friendly, 
unfnendly  and  neutral  forces  -  both  current  and  projected  -  would  be  a 
significant  improvement.  Assessment  tools  were  judged  to  need  improvement. 
The  OA  tools  for  the  physical  domain  are  more  adequate  then  the  information, 
cognitive  or  social  domains. 

As  for  the  Force  Transformation  Process,  it  is  clear  that  operations  analysis, 
even  the  senior  operations  analysts,  are  not  involved  early  in  the  process.  The 
analyst’s  experience,  critical  thinking,  understanding  of  available  models  and 
measures  of  merit  can  be  a  valuable  resource. 

In  addition,  the  OA  community  knows  they  must  help  decision  makers  address 
the  issue:  how  much  is  a  “pound  of  networking  or  horizontal  integration”  worth, 
similar  to  the  old  challenge  associated  with  the  worth  of  a  “pound  of  C4L” 
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Major  Issues  (Continued) 
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•  NCO  and  Coalition  Forces 

♦  Collaboration  among  coalition  nations  remains  insufficient. 

•  Appiying  NCO  to  Actuai  Events 

♦  Need  to  consistently  deploy  and  network  analysts  with 
operational  forces. 

♦  Information  sharing  will  get  more  difficult  due  to  increasing 
reliance  on  SIPRNET  and  similar  activities  and  will  constrain 
collaboration  among  analysts  as  well  as  operators. 


Although  there  was  significant  involvement  and  sharing  of  allied  coimtries’  OA 
capability  to  support  NCO,  it  was  judged  that  collaboration  among  coalition 
nations  remains  insufficient  due  to  multi-level  security,  OA  resources,  and  other 
factors.  Similar  constraints  (e.g.  security)  will  impact  information  sharing 
among  analysts.  The  deployment  and  netting  of  operations  analysts  with 
sufficient  reachback  capabilities  is  a  major  issue  that  must  be  addressed  as  NCO 
concepts  and  tactics  are  developed. 
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Current  “State  of  the  Health” 
for  OA  Process 
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Mftttaiy  Operations  Research  Society 


Over  five  years  ago  at  the  MORS  Mini-Symposium  and  Workshop  on 
“Analyzing  C4ISR  for  2010,”  a  template  for  the  NATO  Code  of  Best  Practice 
for  Assessing  Command  and  Control  was  developed  and  evaluated.  This 
template  was  used  to  evaluate  the  OA  process  for  NCO.  The  associated  view 
graph  documents  the  evaluation.  There  seems  to  be  an  overall  decrease  in  the 
adequacy  of  the  process  due  mainly  to  a  feeling  that  NCO  may  be  more  difficult 
then  C4ISR. 
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NCW Four  Key  Domains 
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-  Current  “State  of  the  HEALTH”  for  Application  of  Operations  Analysis  Process 

Physical  Domain  I 

where  strike,  protect,  and  maneuver  take  place  across  different  environments  | 

||||||||||llllilllllllllllillllllllllllllllllllllilllllllllllllillillllillH 

Information  Domain 

where  information  is  created,  manipulated  and  shared 

illllllllliyillllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllW^ 

H  Cognitive  Domain 

^  where  perceptions,  awareness,  beliefs,  and  values  reside  and 
where,  as  a  result  of  sensemaking,  decisions  are  made 


Each  working  group  was  asked  to  assess  the  “state  of  the  health”  of  the  NCW 
Conceptual  Framework  that  was  discussed  at  the  plenary  session.  The  overall 
qualitative  assessment  is  documented  in  the  associated  chart.  The  physical 
domain  is  assessed  to  be  in  much  better  “health”  from  the  application  of  the  OA 
process  than  the  other  three  domains. 
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•  Findings: 

♦  MOMs  must  both  enable  the  analyst  and  inform  the  client; 
these  may  be  starkly  different  for  NCOA.  MOMs  for  NCOA 
are  a  work-in-progress. 

♦  Survivability-related  MOM  are  increasingly  relevant  to 
military  operations  and  remain  so  for  NCOA  as  well,  at  all 
levels  (MOP,  MOE). 

♦  A  combination  of  MOMs  is  necessary  for  OA  of  networked 
“cause-and-effecf  relationships. 

•  Recommendations: 

♦  Use  findings  of  this  workshop  to  develop  a  TOR  to  structure  a  foliow- 
on  MORS  event  (e.g.,  mini-Symposium)  in  order  to  expose  a  larger 
body  of  analysts  to  NCOA  practice  to  date  (challenges,  initiatives, 
successes,  failures). 


The  findings  and  recommendations  from  the  Measures  of  Merit  Working  Group 
focused  on  the  assessment  that  the  MOMs  for  NCO,  especially  for 
understanding  a  networked  “cause  and  effect”  relationship,  are  a  work-in¬ 
progress.  However,  previous  MOMs  for  the  physical  domains  for  warfighting 
missions  are  well  understood  and  remain  relevant.  A  recommendation  of  the 
MOMs  working  group  as  well  as  others  was  to  conduct  a  future  MORS  session 
to  share  results  of  NCO  related  OA. 
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Processes  and  Methodologies 
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•  Findings: 

♦  The  NCO  framework  emphasizes  “Quality  of  Individual 
Sensemaking”  and  the  “Quality  of  Group  Sensemaking”  versus 
decision  making. 

♦  Many  in  the  NCO  community  assume  that  cognitive  modeling  is 
the  solution  to  modeling  sensemaking  (as  opposed  to  one  of  the 
potential  solutions). 

♦  The  lack  of  processes  and  methodologies  for  OA  using 
sensemaking  and  cognitive  modeling  make  the  NCO  framework 
high  risk. 

•  Recommendations: 

♦  Continue  research  on  sense  making  and  cognitive  modeling 

♦  Consider  revising  the  NCO  framework  to  focus  on  “Quality  of 
Individual  Decision  Making”  (to  enable  decision  superiority). 

♦  Allow  sense  making  and  cognitive  modeling  to  compete  with 
other  modeling  techniques  in  the  implementation  of  NCO 
modeling. 


The  findings  of  the  Process  and  Methodologies  Work  Group  focused  on  the 
NCW  conceptual  framework  and  the  gap  between  sensemaking  and  decision 
making.  It  was  also  suggested  that  the  relationship  between  cognitive  modeling 
and  sensemaking  modeling  needs  to  be  addressed.  Research  in  this  area  was 
recommended,  along  with  a  need  for  the  framework  to  focus  on  the  “Quality  of 
Individual  Decision  Making.” 
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Assessment  Tools 
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•  Findings: 

♦  The  tools  exist,  but  the  data  domain  (both  real  blue  and 
red  data)  is  still  too  stovepiped. 

♦  The  methods  to  change  information  to  knowledge  seem  to 
exist  within  the  models,  but  are  not  well  understood. 


♦  Netting  assessment  tools  is  necessary  but  not  sufficient; 
we  must  learn  to  “do  better  things,”  not  "do  things  better.” 

♦  The  networked  force  must  be  treated  as  a  holistic  entity 
not  as  a  sum  of  discrete  elements. 


•  Recommendations: 

♦  Ensure  assessment  tools  are  used  at  the  “relevant”  scale 
of  operations. 

♦  Adequately  define  the  problem  before  selecting  the 
assessment  tool. 


The  findings  of  the  Assessment  Tools  Working  Group  was  along  three 
dimensions:  1)  “better”  data;  2)  improvement  in  NCO  assessment  tools;  3)  and, 
the  netting  of  assessment  tools  in  order  to  achieve  synergy  in  the  application  of  a 
tool  set. 

A  major  recommendation  of  the  team  was  to  adequately  define  the  problem  and 
MOMs  so  the  “relevant”  scale  of  operations  is  understood  before  selecting  the 
assessment  tool. 
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Force  Transformation  Process 
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•  Findings: 

♦  The  further  we  move  away  from  the  transformation  efforts  that 
are  focused  on  single  service,  tactics,  material  and  entities,  the 
fewer  OA  tools  are  available  to  the  analyst  for  cognitive  and 
social  assessments. 

•  Recommendations: 

♦  Use  NCO  principles  to  organize  and  employ  OA  capabilities  to 
enable  force  transformation. 

♦  Expand  the  traditional  core  competencies  by  using 
interdisciplinary  analysis  teams. 


To  effect  Force  Transformation  it  was  concluded  that  the  further  we  move  away 
from  assessing  transformations  that  are  focused  on  single  service,  tactics  and 
material,  (“old”  missions,  etc.)  the  fewer  OA  tools  are  available  to  the  analyst 
for  information,  cognitive  and  social  domain  related  assessments.  A  major 
recommendation  was  to  organize  or  network  and  deploy  OA  capabilities  in  order 
to  enable  force  transformation  assessment  activities. 
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NCO  and  Coalition  Forces 
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•  Findings: 

♦  OA  analysis  timelines  are  not  fast  enough  to  meet  the 
pace  of  technology  and  DOTMLPF  development. 

♦  OA  is  overly  focused  on  the  military  to  the  exclusion 
of  civilian  agencies. 

♦  Need  to  consider  a  new  level  of  embedded  analysis 
(OA  in-theater  w/data  mining  and  assessment  tools) 
to  provide  real  time  feedback  on  NCO  effectiveness. 

•  Recommendations: 

♦  Improve  the  rigor  of  analysis  of  case  studies  and 
experiments  (hypothesis  testing,  analysis  plans, 
collection  and  dissemination). 


The  NCO  and  Coalition  Forces  Working  Group  found  that  the  OA  timelines  are 
not  fast  enough  to  keep  pace  with  technology  and  other  developments.  The 
experience  of  the  UK’s  embedded  OAs  needs  to  be  assessed  and  maybe 
followed.  However,  the  inclusion  of  civilian  agencies  also  must  be  addressed. 

Based  on  coalition  forces  experience,  it  was  recommended  that  OA  be  made 
more  proactive  and  rigorous  (hypothesis  testing,  analysis  plans,  etc.)  by 
involving  OAs  as  the  problems  for  the  case  studies  and  experiments  are  defined. 
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Applying  NCO  to  Actual  Events 
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•  Findings: 

♦  Lack  of  joint  and  some  service  doctrine  to  incorporate  analysis  in  a  JTF 
limits  OA  support  of  NCO. 

♦  Reliance  on  non-embedded  analysts  makes  OA  an  afterthought  in 
JTFs. 

♦  OA  currently  limited  by  available  and  valid  data. 

Dynamic  environment;  the  data  is  not  clean  or  consistent 
Warfighting  systems  don’t  output  OA-ready  data 
■>  Data  collection  must  be  pre-planned  due  to  the  numerous  organizations 
involved  and  their  focus  on  the  real-world  operation 

♦  Time  and  data  constraints  limit  the  tool  set. 

•  Recommendations: 

♦  Develop  products  and  processes  to  facilitate  NCO  information  sharing 
in  order  to  network  analysts  with  collaborative  capabilities. 

♦  Develop  methodology/tools  to  enhance  OA  support  within  time 
constraints  of  the  actual  event. 

♦  Develop  joint  doctrine  to  plan  and  use  OA  support  for  NCO. 


When  assessing  applications  of  OA  to  NCO  actual  events,  it  was  found  that, 
because  policies  and  doctrine  to  incorporate  analysis  is  lacking,  along  with  the 
non-use  of  embedded  analysts,  OA  becomes  an  afterthought.  If  these  findings 
and  the  data  domain  is  improved  then  the  assessment  of  actual  NCO  events  will 
be  improved.  Recommendations  focused  on  developing  doctrine,  plans  and 
processes  to  network  OAs  and  include  them  early. 
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‘'Actionable  "  Recommendations 
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•  Establish  a  “business  model”  that  enables  services 
to  conduct  operations  analysis  in  support  of  “born 
joint”  NCO  concepts  and  that  is  underpinned  by  a  set 
of  approved  tools  and  data. 

•  Provide  the  capability  to  capture  and  share  results 
and  data  of  experiments,  training,  analysis  and 
experience.  One  way  to  do  this  is  to  Create  center(s) 
of  excellence  within  services  and  JFCOM  for  NCO 
assessments. 

•  Enable  (thru  agreements,  business  practices)  the 
networking  of  analysts  working  NCO  and  provide 
reach  back  for  tools,  data  and  previous  studies. 

•  Conduct  a  follow  on  MORS  mini-symposium  to  share 
NCO  analysis  and  experimentation  results. 


An  effort  to  synthesize  the  findings  and  recommendations  into  a  short  list  of 
“actionable”  recommendations  was  attempted  for  the  first  time.  This  list  is  in 
the  associated  chart.  They  focus  on  creation  of  processes  to  conduct  joint  OA 
for  NCO,  to  link  analysis  centers  of  excellence  and  net  deployed  OAs  with 
adequate  reachback.  In  addition,  the  support  of  a  follow  on  MORS  session  was 
recommended. 
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•  Good  News 

♦  OA  process  developed  in  the  Industrial  Age  has  applicability  to 
Network  Centric  Operations  (NCO)  in  the  Information  Age. 

♦  We  are  beginning  to  understand  the  practices  that  should  be 
followed  to  perform  credible  NCO  analyses. 

From  attrition  based  to  effects  based 

♦  We  know  there  are  NCO  considerations  that  continue  to  present 
chalienges  (i.e.,  social  domain, ...). 

•  Bad  News 

♦  Currently,  we  do  not  do  a  satisfactory  job  in  documenting, 
sharing  and  learning  from  NCO  analyses  performed. 


The  Synthesis  Team  discerned  that  news  on  OA  support  to  NCO  ran  the  gamut 
from  good  to  bad  to  worse  to  better. 

The  good  news  is  that  the  OA  process  developed  in  the  Industrial  Age  has 
applicability  to  Network  Centric  Operations  (NCO)  in  the  Information  Age.  We 
are  starting  to  understand  the  practices  that  should  be  followed  to  perform 
credible  NCO  analyses.  In  addition,  we  know  there  are  NCO  considerations  that 
remain  challenges. 

The  bad  news  is  that  we  do  not  do  a  satisfactory  job  in  documenting,  sharing  and 
learning  from  NCO  analyses  performed. 
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•  Worse  News 

♦  With  the  arrival  of  terrorism,  other  asymmetric  threats, 
rogue  nations,  and  homeland  defense,  the  NCO 
assessment  problem  and  Force  Transformation,  are 
substantially  more  difficult  and  more  urgent. 

•  Better  News 

♦  If  the  recommendations  of  this  workshop  are 
implemented,  they  will  provide  a  basis  for  the 
Operations  Analysis  Process/Community  to  address 
the  most  critical  analytical  challenges  to  support  NCO 
and  Force  Transformation. 


The  worse  news  is,  that  with  the  arrival  of  terrorism,  other  asymmetric  threats, 
rogue  nations,  and  homeland  defense,  the  NCO  assessment  problem  and  Force 
Transformation  are  substantially  more  difficult  and  more  urgent. 

Although  these  are  daunting  challenges,  there  is  some  better  news  that  is  implicit 
in  this  workshop  itself  If  the  recommendations  of  this  workshop  are 
implemented  successfully,  they  will  provide  a  basis  for  the  Operations  Analysis 
process  and  community  to  address  the  most  critical  analytical  challenges  to 
support  NCO  and  Force  Transformation. 
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MORS  Special  Meeting  Acronyms 

MORS  Workshop: 

Operations  Analysis  Support  to  Network  Centric  Operations 


ABCA 

ACIDC 

ADF 

ADOC 

AFSAA 

AoA 

AU 

BAH 

C2 

C2P2 

C3 

C4ISR 

CA 

CAA 

CBA 

CBMARTD 

CCRP 

CD&E 

GENICOM 

CENTRK 

CFLCC 

CJTF-7 

CNO 

COBP 

COCOM  TSCPs 

CONOPS 

COTS 

DAST 

DoD 

DOS 

DOTMLPF 

DPG 

EBO 

EBR 

FCS 

FOI 

FS 

HPTs 

HUMINT 


American,  British,  Canadian,  Australian  Armies  Standardization  Program 
WG  5  ^nited  Kingdom) 

Australian  Defence  Force 

Air  Defense  Operations  Center 

Ar  Force  Studies  and  Analyses  Agency 

Analysis  of  Alternatives 

Australia 

Booz  Allen  Hamilton 
Command  and  Control 

Army  (WG  6)  Command  and  Control  Protection???? 

Command,  Control  and  Communications 

Command,  Control,  Communications,  Computers,  Intelligence, 

Surveillance  and  Reconnaissance 
Canada 

Center  for  Army  Analysis 
Capabilities  Based  Acquisition 
WG  5  (United  Kingdom) 

Command  and  Control  Research  Program 
Concept  Development  and  Experimentation 
United  States  Central  Command 

Combined  Enterprise  Regional  Information  Exchange  System 
Coalition  Forces  Land  Component  Command 
Combined  Joint  Task  Force  (NATO) 

Chief  of  Naval  Operations 
Code  of  Best  Practices 

Combatant  Command  Technical  Control  Systems  Programs 

Concept  of  Operations 

Commercial  Off  The  Shelf 

Deployable  Analysis  Support  Team 

Department  of  Defense 

Department  of  State 

Doctrine,  Organization,  Training,  Material,  Leadership,  Personnel,  Facilities 

Defense  Planning  Guidance 

Effects  Based  Operations 

Evidence  Based  Research 

Future  Combat  System 

Swedish  Defense  Research  Agency 

Fellow  of  the  Society  (MORS) 

High  Priority  Targets 
Human  Intelligence 
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lA 

lED 

ffiR 

10 

ISR 

IT 

JEFX 

JFCOM 

JLLT 

JMEM 

JNTC 

JROC 

JTF 

JUEP 

JWARS 

MDMP 

MoD 

MOEs 

MOMs 

MOOs 

MOPs 

MORS 

MORSS 

NATO 

NC 

NC3A 

NCO 

NCOA 

NCOW 

NOW 

NEC 

NGO 

NR 

NMS 

NPS 

OA 

OEF 

OFT 

OIF 

OOTW 

OPTEMPO 

ORD 

ORSA 

OSD 

PK 

SEAS 


Information  Assurance 

Improvised  Explosive  Devices 

Information  Exchange  Requirement 

Information  Operations 

Intelligence,  Surveillance,  Reconnaissance 

Information  Technology 

Joint  Expeditionary  Force  Experimentation 

US  Joint  Forces  Command 

Joint  Lessons  Learned  Team 

Joint  Munitions  Effectiveness  Manual 

Joint  National  Training  Capability 

Joint  Requirements  Oversight  Council 

Joint  Task  Force 

WG  5  (United  Kingdom) 

Joint  Warfare  System 
Military  Decision  Making  Process 
Ministry  of  Defense 
Measures  of  Effectiveness 
Measures  of  Merit 

Measures  of  Objectives  or  Measures  of  Outcomes 

Measures  of  Performance 

Military  Operations  Research  Society 

Military  Operations  Research  Society  Symposium 

North  Atlantic  Treaty  Organization 

Network  Centric 

NATO  C3  Agency 

Network  Centric  Operations 

Network  Centric  Operations  Analysis 

Network  Centric  Operations  and  Warfare 

Network  Centric  Warfare 

Network  Enabled  Capability 

Non  Governmental  Organization 

Networks  Intelligence  and  Information 

National  Military  Strategy 

Naval  Postgraduate  School  , 

Operations  Analysis 
Operation  Enduring  Freedom 
Office  of  Force  Transformation 
Operation  Iraqi  Freedom 
Operations  Other  Than  War 
Operations  Tempo 

Operational  Requirements  Document 
Operations  Research  Society  of  America 
Office  of  the  Secretary  of  Defense 
Probability  of  Kill 

System  Engineering  Analysis  Simulation 
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SIAP 

Single  Integrated  Air  Picture 

SJFHQ 

Standing  Joint  Force  Headquarters  Core  Element 

SoS 

Systems  of  Systems 

SW 

Sweden 

TOR 

MORS  Term  of  Reference 

TPG 

Technology  Planning  Guide 

TRAC 

TRADOC  Analysis  Center 

TRADOC 

Training  and  Doctrine  Command 

TST 

Time  Sensitive  Target 

TTCP 

Test  Training  Certification  Plan 

TTP 

Tactics,  Techniques,  and  Procedures 

UAV 

Unmanned  Aerial  Vehicle 

UK 

United  Kingdom 

USA 

United  States  Army 

USA/US 

United  States  of  America 

USAF 

United  States  Air  Force 

USMA 

United  States  Military  Academy 

USMC 

United  States  Marine  Corps 

USN 

United  States  Navy 

WIN-T 

Warfighter  Information  Network  -  Tactical 
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MORS  Special  Meeting  Terms  of  Reference 

MORS  Workshop: 

Operations  Analysis  Support  to  Network  Centric  Operations 

1.  Background  - 

As  Network  Centric  Operations  (NCO)  are  being  developed,  assessment  needs  vs.  capability 
gaps  are  being  discussed  and  options  considered  by  the  US  Armed  Forces.  In  these  early 
stages,  an  assessment  of  Operations  Analysis  to  support  Network  Centric  Operations  needs  to 
be  conducted. 

For  instance,  functional  requirements  and  metrics  need  to  be  developed.  Some  MOEs  and 
MOPs  have  been  developed  for  some  mission  areas,  but  not  for  the  higher  level  command 
and  control  functions.  Also,  what  are  the  analytic  processes  to  assess  Network  Centric 
Operations  for  PPBS  and  joint  experimentation?  What  methodologies  will  enable 
quantification,  insight  and  understanding  on  NCO  Concepts  of  Operations?  What  Modeling 
&  Simulation  is  available  to  support  assessments  of  NCO  Concepts  and  what  are  the 
campaign  model  limitations? 

Additionally,  our  growing  Operational  and  Systems  Architectures  heed  to  be  leveraged  as  we 
imderstand  bandwidth  requirements.  How  can  NCO  be  applied  in  the  Force  Transformation 
Process?  How  will  it  shape  force  structure  and  the  concepts  of  operations?  What  are  the 
inter-relationships  between  information  and  weapons  systems? 

Furthermore,  are  there  interoperability  issues  among  the  Joint,  Allied  and  Coalition  Forces? 
How  are  Allied  &  Coalition  Forces  using  operations  analysis  to  support  NCO?  What  are 
their  lessons  learned?  How  do  we  stay  in  step  with  the  Coalition  Forces?  What  do  we  do  for 
those  Coalition  Forces  that  will  not  /  cannot  participate  in  the  network?  What  about  with  US 
Agencies,  such  as  Homeland  Security,  Justice,  etc.? 

Finally,  how  can  effects-based  operations  be  applied  to  NCO?  How  effective  are  our 
Information  Warfare  operations?  When  is  it  more  effective  to  achieve  non-lethal  vs.  lethal 
effects?  How  do  we  determine  how  effective  Information  Warfare  is?  What  are  the  co¬ 
evolving  landscapes  in  C4ISR?  Are  there  better  data  practices?  How  do  cognitive  and 
behavioral  factors  influence  command  and  control? 

This  Workshop  will  research  and  examine  the  above  questions,  using  the  applicable  NCO- 
related  findings  of  the  recently  completed  MORS  Special  Meetings  on  Effects-Based 
Operations,  Information  Operations,  C4ISR,  Data  Practices,  Cognitive  and  Behavioral 
Factors  Influencing  Command  and  Control,  and  Decision  Support. 
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Many  organizations  are  interested  and  involved  in  emplo3dng  and  analyzing  Network  Centric 
Operations.  The  analytic  community  needs  to  support  the  military  services  with  assessments 
as  Network  Centric  Operations  is  being  developed.  The  community  should  play  a  leadership 
role  in  creating/refining  these  needed  metrics,  processes,  methodologies,  models  and 
simulations.  The  community  should  share  efforts,  successes  and  failures  in  the  key 
capabilities. 

The  goal  of  this  meeting  is  to  provide  an  opportimity  to  bring  a  multi-disciplined  team  of 
analysts,  operators  and  engineers  from  those  organizations  together  to  share  their  work, 
develop  a  common  view  of  the  state  of  practice,  expose  members  of  the  broader  analytic 
community  to  their  needs,  identify  shortfalls  and  potential  solutions. 

There  are  several  overarching  issues  e‘ach  working  group  will  consider.  They  are: 

•  What  is  the  state  of  the  practice?  Identifying  key  issues  and  shortfalls  - 

-  What  is  the  guidance  for  best  principles  and  practices  relating  to  NCO 
assessments? 

-  What  area(s)  do  we  need  to  understand  better  than  the  current  knowledge  levels? 

-  What  area(s)  do  we  need  to  prioritize  higher  to  xmderstand  sooner? 

-  What  are  the  operational  data  concerns  (i.e.,  observables,  validation,  etc.)? 

-  What  are  the  operations  analysis  competency  development  and  knowledge 
sharing  concerns? 

-  What  are  the  cultural  issues? 

•  What  is/should  be  the  context  and  relationships  between  the  Operations  Analysis 
community  and  the  stakeholder  (those  organizations  working  with  NCO)  needs? 

•  What  are  the  examples  of  how  OA  has  been  applied  to  support  NCO? 

-  What  are  the  future  challenges? 

-  How  do  NCO  assessments  fit  in  the  overall  OA  process? 

■  Define  the  problem 

■  Determine  the  appropriate  Measures  of  Merit 

■  Select  a  scenario  set 

■  Determine  the  representative  operational  situations/CONOPS 

■  Determine  the  models 

■  Conduct  the  analysis 

■  Evaluate/commimicate  results 

■  Assess/feedback 
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•  Do  the  results  of  recently  completed  MORS  Workshops  apply  to  NCO?  If  so,  how? 

-  Analyzing  Effects-Based  Operations 

-  Operations  Research  Methods  for  Information  Operations 

-  Advancing  C4ISR  Assessment 

-  Improving  D^ense  A  nalysis  through  Better  Data  Practices 

-  How  Cognitive  and  Behavioral  Factors  Influence  Command  and  Control 

-  Decision  Aids/Support  to  Joint  Operations  Planning 

•  How  will  the  other  U.  S.  Agencies  (i.e.,  HLS,  Justice,  etc.)  collaborate  in  an  NCO 
environment?  Has  any  OA  been  conducted  to  incorporate  the  Agencies  with  the 
Service’ s  NCO  environments? 

•  Other  considerations  (primarily  for  WGs  4,  5  &  6).  What  analyses  has  been 
completed  providing  recommendations : 

-  To  improve  known  transmission  and  bandwidth  issues  and  limitations? 

-  To  minimize  fratricide,  IFF,  Combat  ID  issues? 

-  To  fuse  data? 

The  overall  objectives  of  this  special  meeting  on  Operations  Analysis  Support  to  Network 
Centric  Operations  will  be  to  provide  an  assessment  and  a  roadmap  to  revitalize  the  state  of 
the  analytical  practice  and  to  recommend  priorities  for  any  initiatives  identified.  Some 
specific  objectives  for  the  Working  Groups  include  recommending: 

•  Functional  requirements  and  metrics  that  need  to  be  developed  /  refined 

•  Analytical  processes  and  methodologies  to  optimize  and  evaluate  NCO 

•  Modeling  and  simulation  to  support  NCO  assessments 

•  How  NCO  assessments  can  be  applied  in  the  Force  Transformation  Process 

•  NCO  assessments  collaboration  among  the  Joint,  Allied  and  Coalition  Forces 

•  How  quick  turnaround  analyses  of  NCO  enables  an  event 

3.  Approach  - 

a.  1  Day  -  Mini-Symposium  -  The  meeting  will  commence  with  a  mini-symposium  format 
that  will  include  operational  based  discussions  as  well  as  progress  to  date.  The  purpose 
of  this  portion  is  to  bring  all  participants  up  to  speed  on  the  state  of  the  practice  and  frame 
the  analytical  challenges  and  issues  for  the  working  groups. 

(1)  Keynote  Presentation  (~  1  hour  presentation  with  15  minutes  for  Q&A)  -  From  a 
high,  Joint-level  perspective: 

(a)  Define  and  provide  a  brief  background  on  Network  Centric  Operations 

(b)  Identify  the  overriding  challenges  the  analytical  community  needs  to  address 

(c)  Provide  an  overview  of  the  progress  to  date  using  operations  analysis  to  support 
NCO,  including  any  problems  and  paradoxes 
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(2)  Service  Presentations  (total- 1  V2  hours  with  ~  15  minutes  for  each  Service  and -- 15 
minutes  available  for  questions)  - 

(a)  Presentations  —  (each  Service  and  1  Coalition  Country;  ~  15  minutes  each) 

-  Provide  an  overview  of  the  challenges  and  progress  to  date  using  operations 
analysis  to  support  NCO  (with  examples) 

-  Highlight  context,  issues,  what  is  needed  from  the  analytical  community 

-  Identify  some  of  the  broad  analytical  and  difficult  questions  , 

(b)  O&A  -  After  all  presentations,  the  remaining  time  (~  15  minutes)  will  be  for 
questions 

(3)  Framework  Presentations  (~  1  V2  hours)  - 

(a)  An  Overview  of  NCO  Differences  (~  45  minutes)  -  Comparing  and  contrasting 
the  NCO  differences: 

-  Among  the  US  Services 

-  With  the  US  Agencies  (i.e.,  HLS,  Justice,  etc.) 

-  With  the  Coalition  Forces 

-  Highlighting  OA  examples,  where  appropriate 

(b)  What  is  Network  Centric  Operations?  (~  20  minutes) 

-  After  the  attendees  have  listened  to  the  Keynote,  Service  and  NCO  Differences 
presentations,  this  is  a  brief  to  focus  the  Working  Group  discussions 

(c)  What  is  the  Operations  Analysis  Process?  (~  30  minutes) 

-  This  is  another  brief  to  focus  the  Working  Group  discussions 

-  A  review  of  the  OA  process  to  see  how  OA  “fits”  in  the  six  Working  Groups 

(4)  Guidance  Presentations  (~  1  ‘A  hours)  - 

(a)  Workshop  Introduction  Brief  -  by  the  Special  Meeting  Co-Chairs  (~  20  minutes) 

-  Discuss  the  Working  Group  objectives 

-  Highlight  the  overarching  analytical  issues 

-  Provide  a  schedule  overview  for  the  Workshop 

-  Introduce  the  Working  Groups  and  their  Chairs,  Co-Chairs  &  Advisors 

(b)  Technical  Chair  Comments  and  Guidance  (~  10  minutes) 

(c)  Reports  on  previously  completed  related  MORS  Workshops  (listed  above), 
highlighting  any  relationships  to  NCO  (~  50  minutes  ~  10  minutes  each) 
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(d)  Synthesis  Group  Overview  -  by  Synthesis  Group  Chair  (~1 0  minutes) 

-  Discuss  the  role  of  the  Synthesis  Group 

-  Highlight  what  input  is  desired  from  the  attendees 

-  Introduce  the  Synthesis  Group 

b.  2"*^  &  3^^  Day  -  Workshop  -  The  Mini-Symposium  will  be  followed  by  a  two-day 
workshop  where  participants  will  meet  in  working  groups  to  further  examine  specific 
topics,  including  discussing  the  overarching  issues  of  the  Workshop.  Working  groups 
will  prepare  a  report  on  their  activities  to  present  to  other  workshop  participants  at  the 
last  session  of  the  workshop.  To  focus  the  discussion  in  each  of  the  working  groups,  a 
select  group  of  people  will  be  requested  to  prepare  and  present  papers.  The  workshop 
attendees  will  be  organized  into  six  working  groups  plus  a  synthesis  group.  The  working 
group  structure  is  detailed  below. 

(1)  WG  1 :  Measures  of  Merit  —  Using  the  OSD  project  on  NCW  Framework  and  Dr. 
Albert’s  “Maturity  Matrix”  to  measure  progress  of  the  network  as  a  foundation,  this 
working  group  will  examine  the  functional  requirements  and  metrics  being  developed 
for  Network  Centric  Operations.  Some  potential  discussion  points  include: 

(a)  What  are  the  issues  from  previous  work  on  NCO  metrics?  Are  any  of  these  issues 
not  defined?  If  so,  which  ones? 

(b)  What  is  the  status  of  not  only  the  network  MOPs,  but  also  the  transformation 
MOEs?  Network  MOPs  may  not  capture  the  synergy  associated  with  being  able 
to  network. 

(c)  The  Army’s  Future  Combat  System  and  the  Navy’s  FORCEnet  have  metrics. 

-  Are  they  consistent? 

-  Are  they  inclusive? 

-  Are  they  adequate? 

(d)  What  about  the  Key  Performance  Parameters  (KPPs)?  What  should  they  be? 

(e)  Can  the  previously  proposed  NCO  measures  be  “audited”  to  determine  where  the 
gaps  are? 

(f)  Can  “traditional”  metrics  make  a  contribution  to  evaluating  NCO?  If  so,  how?  If 
not,  why  not? 

(g)  What  is  the  frame  of  reference  for  examining  the  relationship  among  NCO,  Rapid 
Decisive  Operations  (RDO)  and  Effects  Based  Operations  (EBO)  metrics? 

Define  the  common  areas.  By  doing  so,  this  may  enhance  the  ability  to  use  the 
previous  work  on  metrics  for  RDO  and  EBO  that  applies  to  NCO. 

(h)  How  do  you  determine  and  link  metrics  across  the  physical,  information, 
cognitive  and  social  domains?  What  are  the  metrics  in  each  domain? 

(i)  What  are  observable  and  measurable  metrics  vs.  intangible  metrics? 

(j)  How  are  the  new  metric  hierarchies  linked  to  combat  power  and  outcomes? 
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(2)  WG  2:  Processes  and  Methodologies  -  This  working  group  will  examine  analytical 
processes  and  methodologies  to  optimize  and  evaluate  Network  Centric  Operations. 
Some  potential  discussion  points  include: 

(a)  Analysis  process  changes  needed  in  order  to  assess  NCO  in  light  of  the  forces  of 
transformation,  e.g.,  EBO,  capabilities-based  acquisition,  spiral  development, 
increasing  JFCOM  role  and  “joint”  task  force  capabilities  perspective,  and 
increasing  involvement  with  NGOs,  international  entities,  and  OOTW. 

-  What  processes  changes  are  needed  to  better  reflect  the  importance  of  joint 
experimentation  (assessing  concepts)  and  the  potentially  decreasing  need  for 
traditional  mission  analysis,  and  to  involve  stakeholders  more? 

-  What  are  the  emerging  implications  for  the  methodologies  and  tools  that 
support  these  processes,  e.g.,  emerging  need  for  mini-wargames  to  supplant  or 
supplement  campaign  level  models? 

-  Given  the  analytical  community’s  current  capability  in  assessing  NCO,  what 
process  changes  could  be  implemented  in  the  interim  and  what  needs  to  take  a 
longer-term  perspective,  e.g.,  process  improvement  road  map? 

(b)  The  methodologies  to  enable  quantification,  insight  and  understanding  of  NCO 
CONOPS  and  information  age  paradigms. 

(3)  WG  3:  Assessment  Tools  -  This  working  group  will  examine  and  recommend  the 
modeling  and  simulation  to  support  assessments,  i.e.,  POM  to  system  engineering  to 
experimentation  of  NCO  Concepts,  including  the  role  of  wargames,  field  experiments 
and  exercises.  Some  potential  discussion  points  include: 

(a)  The  campaign  model  limitations 

-  Currently  assume  “perfect  knowledge  /  ISR” 

-  Recommended  solution  (FORCEnet,  CEC,  etc.)  goals  are  to  deliver  “near- 
perfect  environment”  . . .  shows  no  delta  in  performance 

(b)  Representation  of  C2,  human,  organizational  behaviors  and  interactions 

-  Trust  in  netted  environment 

-  History  of  “mistakes” 

(c)  The  representative  hierarchy  of  models 

(4)  WG  4:  Force  Transformation  Process  -  This  working  group  will  examine  how  using 
OA  in  support  of  Network  Centric  Operations  can  be  applied  in  the  “Force 
Transformation  Process”.  Some  potential  discussion  points  include: 

-  NCO  assessments  shaping  force  structure  and  the  concepts  of  operations 

-  Operational  and  system  architectures  required 
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-  NCO  doctrine  towards  transformation 

-  Inter-relationships  between  information  and  weapons  systems 

(5)  WG  5:  NCO  and  Coalition  Forces  -  This  working  group  will  examine  applying  OA 
to  Network  Centric  Operations  and  interoperability  among  the  Joint,  Allied  and 
Coalition  Forces.  Some  potential  discussion  points  include: 

-  In  addition  to  the  Joint  Forces,  how  are  other  (i.e..  Allied  &  Coalition)  Forces 
using  operations  analysis  to  support  NCO? 

-  What  are  their  lessons  learned? 

-  What  collaboration  assessments  have  been  accomplished  among  the  Joint  and 
Coalition  Forces? 

-  How  do  the  Joint  Forces  stay  in  step  with  the  Allied  &  Coalition  Forces? 
Training,  exercises,  policy?  Information  sharing? 

-  How  do  the  Joint  Forces  maintain  the  relationship  with  those  Coalition  Forces 
that  are  not  /  can  not  get  on  the  network? 

(6)  WG  6:  Applying  NCO  to  an  Actual  Event  —  This  working  group  will  examine  how 
applying  OA  to  Network  Centric  Operations  enables  time  critical  events.  Some 
potential  discussion  points  include: 

-  What  is  a  collaborative  target? 

-  In  what  length  of  time  is  “critical”? 

-  How  can  quick-tumaroxmd  analyses  support  the  event? 

-  What  type(s)  of  analysis(es)  is(are)  recommended? 

-  The  different  ways  the  Air  Force  used  OA  and  lessons  learned  in  recent  events 
to  reduce  the  time  from  4  hours  to  45  minutes  (with  a  goal  of  2-3  minutes) 

(7)  Synthesis  -  The  synthesis  group  will  bring  together  the  work  of  the  six  working 
groups  and  develop  overall  assessment/recommendations  from  the  analysis 
community  for  the  individual  service  operations  analysts  to  consider. 

4.  Agenda  -  (tentative) 

Dav/Time  Activity 

Monday,  26  January  2004 

1700  Working  Group  Chair  and  Co-Chair  Warm-Up  Session 

Tuesday,  27  January  2004 

0700  Registration  and  Continental  Breakfast 

0800  MORS  President’s  Welcome 

0805  Welcome  by  Host 

0810  Proponent  Welcome 
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0820  Workshop  Overview 

0845  Keynote  Presentation 

1000  Break 

1030  Service  Presentations 

1 200  Lunch  in  Working  Group  Rooms  -  WG  Introductions 

1315  Framework  Presentations 

1500  Break 

1530  Guidance  Presentations 

1700-1830  Mixer 

Wednesday,  28  January  2004 

0715  Continental  Breakfast 

0800  Working  Group  Session#! 

0945  Break 

1015  Working  Group  Session  #2 

1200  Lunch  in  Working  Group  Rooms 

1300  Working  Group  Session  #3 

1445  Break 

1515  Working  Group  Session  #4 

1700  WG  Chair,  Co-Chair  and  Synthesis  Group  Daily  Wash  Up 

Thursday,  29  January  2004 

0715  Continental  Breakfast 

0800  Working  Group  Session  #5 

0945  Break 

1015  Working  Group  Session  #6 

1200  Lunch  in  Working  Group  Rooms 

1330  Working  Group  Outbriefs:  WG  1, 2,  3  &  4 

1500  Break 

1530  Working  Group  Outbriefs:  WG  5,  6  &  Synthesis 

1630  Workshop  Wrap-Up 

1700  Adjourn  Workshop 

Friday,  30  January  2004 

0800  Working  Group  Chairs  complete  Working  Group  Annotated  Briefings 

1200  Adjourn  Workshop  (for  Chairs) 
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The  tentative  utilization  for  the  working  group,  sessions  will  be: 

(1)  Day  1  Working  Group  Lunch  -  Kickoff:  Introduction,  agenda,  issues  &  goals;  and 
Provide  context  to  orient  WG  participants  for  discussion  and  debate 

(2)  Working  Group  Session  #  1  -  Technical  Papers  Session  #1 

(3)  Working  Group  Session  #  2  -  Technical  Papers  Session  #2 

(4)  Day  2  Working  Group  Lunch  -  As  required  (Consider:  Speaker  to  address  frontier  issues 
on  WG  topic) 

(5)  Working  Group  Session  #  3  -  Frame  WG  response  &  collect  issues  (brainstorming) 

(6)  Working  Group  Session  #  4  -  Characterize  OA  Support  and  assess  gaps  /  shortfalls 

(7)  Working  Group  Session  #  5  -  Recommend  strategies  and  roadmaps 

(8)  Working  Group  Session  #  6  -  Refine  ideas,  arguments,  capture  WG  debate,  etc. 

(9)  Day  3  Working  Group  Lunch  -  Complete  presentation  for  WG  Outbrief 

5.  Attendees  - 

a.  Attendance  will  be  by  invitation  only.  Attendees  will  include  invited  experts  from  OSD, 
all  Services,  the  Joint  Staff,  University  Affiliated  Research  Centers,  Federally  Funded 
Research  and  Development  Centers,  operational  commanders,  DoD  contractors  and 
others,  including  representatives  from  our  Allied  /  Coalition  Analytical  Communities . 
Workshop  chairs  will  control  membership  of  their  sessions  in  conjunction  with  the 
Organizing  Committee.  Attendance  will  be  limited  to  200  people. 

b.  Working  Groups  (WGs)  will  be  led  by  a  Chair,  one  to  three  Co-Chairs  and  an  Advisor. 
This  leadership  group  will  be  comprised  of  all  MORSians  or  a  combination  of  MORSians 
and  Subject  Matter  Experts  (SMEs).  The  responsibilities  of  this  team  include: 

(1)  Chair - 

(a)  Dynamic  individual  that  is  a  SME  in  the  WG  topic 

(b)  Solicits  analysts  and  operators  in  the  field  to  participate  in  the  WG 

(c)  Guides  the  WG  during  the  Workshop 

(d)  Challenged  to  provide  the  “substance”  of  the  special  meeting  WG 

(e)  Develops  the  WG’s  final  product 

(2)  Co-Chair  -  Individual  interested  in  WG  topic;  assist  Chair  as  Chair  requests 

(3)  Advisor  -  Individual  that  is  a  SME  in  the  operational  side  of  the  WG  topic;  assists 
Chair  in  WG  membership,  provides  perspective  during  Workshop,  and  assists  Chair 
as  requested 

c.  Another  key  group  of  individuals  during  the  Workshop  is  the  Synthesis  Group.  This 
group  will  provide  representation  to  each  of  the  WGs  and  assist  the  Workshop  Chairs 
consolidate  the  working  group  results  and  develop  overall  assessment/recommendations 
from  the  analysis  community  for  the  individual  service  operations  analysts  to  consider. 
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6.  Products  - 


Several  products  will  be  generated  from  the  workshop: 

•  An  Executive  Summary  in  the  form  of  a  text  document  and  a  scripted  briefing  for  the 
MORS  Sponsors  addressing  the  workshop  objectives,  findings,  conclusions  and 
recommendations  will  be  offered  within  30  days. 

•  A  proceedings  document  containing  summaries  of  all  sessions  and  annotated  copies  of 
appropriate  briefing  slides  and  presentations. 

•  An  article  summarizing  the  meeting  and  its  findings  will  be  produced  and  submitted  to 
PHALANX  in  time  for  the  next  deadline  after  the  meeting. 

•  A  general  session  presentation  will  be  made  at  the  72"^  MORSS. 

7.  Milestone  Table  -  See  the  OA  Support  to  Network  Centric  Operations  Plan  of  Actions  & 
Milestones 

8.  Proponents  - 

Director,  Assessment  Division  (N8 1),  Office  of  the  Chief  of  Naval  Operations 

9.  Planning  and  Organizing  Committee  - 

General  Co-Chairs:  Dennis  Baer,  Northrop  Grumman  IT 

Kirk  Michealson,  Lockheed  Martin  ISS/AC-T 

Technical  Co-Chairs:  Dr.  Richard  Hayes,  Evidence  Based  Research,  Inc. 

Dr.  David  Alberts,  OSD(NII) 

Dr.  Jerry  Kotchka,  FS,  Lockheed  Martin  ISS/AC-T 
CAPT  Jeff  Kline,  NPS 

Michael  Bauman,  FS,  TRAC  Ft.  Leavenworth 
David  Garvey,  Alidade  Incorporated 
Dr.  Stuart  Starr,  FS,  The  MITRE  Corporation 
Dr.  Russ  Richards,  JFCOM  (MITRE) 

Dr.  Greg  Parnell,  FS,  USMA 
Prof  Bill  Kemple,  NPS 
Bob  Manke,  Naval  Undersea  Warfare  Center 

Site  Coordinator:  Booz  Allen  Hamilton 

Administrative  Coordinators:  Brian  Engler,  Executive  Vice-President,  MORS 

Natalie  Kelly,  Vice-President  for  Admin,  MORS 

MORS  Bulldog:  Dean  Free,  Anteon 


Synthesis  Chair: 

Group: 
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Working  Group  Chairs: 

WGl  -  Measures  of  Merit: 

Chair  -  Jim  Jacobs,  Northrop  Grumman  IT 
Co-Chair  -  Corinne  Wallshein,  AFSAA 

Advisor  -  Dr.  Kimberly  Holloman,  Evidence  Based  Research,  Inc. 

WG  2  -  Processes  &  Methodologies: 

Chair  -  Dr.  Daniel  Maxwell,  Evidence  Based  Research,  Inc. 
Co-Chair  -  Ralph  Klingbeil,  Naval  Undersea  Warfare  Center 
Co-Chair  -  Jeffrey  Vick,  Boeing  Military  Studies  &  Analysis 
Advisor  -  Graham  Mathieson,  UK  Defence  Science  &  Tech  Lab 

WG  3  -  Assessment  Tools: 

Chair  -  Steven  Beres,  Evidence  Based  Research,  Inc. 

Co-Chair  -  Dr.  Mark  Youngren,  The  MITRE  Corporation 
Co-Chair  -  Ben  Taylor,  UK  Defence  Science  &  Tech  Lab 
Co-Chair  -  TJ  Ferrell,  Boeing  Military  Studies  &  Analyses 
Advisor  -  LtCol  Gregory  McIntyre,  JWARS,  OSD(PA&E) 

WG  A  — Force  Transformation  Process: 

Chair  -  Bob  Gregg,  Whitney,  Bradley  &  Brown,  Inc. 

Co-Chair  -  Duncan  Tailby,  JFCOM  -  Analysis  Division 
Advisor  -  Dr.  John  Hanley,  IDA 

WG  5  -  NCO  &  Coalition  Forces: 

Chair  -  COL  Pat  Vye,  ODUSA(OR) 

Co-Chair  -  Ray  Christian,  Naval  Undersea  Warfare  Center 
Co-Chair  -  Joe  Quartararo,  JFCOM  —  Analysis  Division 
Co-Chair  -  Alan  Cowdale,  UK  Defence  Science  &  Tech  Lab 
Advisor  -  Walt  Perry,  RAND 

WG  6  -  Applying  NCO  to  Actual  Events: 

Chair  -  MAJ  Sean  Deller,  JFCOM,  J9,  Lessons  Learned 
Co-Chair  -  MAJ  Greg  Petrick,  Hanscom  AFB,  Analysis  Division 
Advisor  -  COL  Steven  Mains,  JFCOM,  J9,  Lessons  Learned 
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Sponsor/Service  Reps; 
Air  Force: 

Army: 

Navy: 

Marine  Corps; 
Joint  Staff: 

OSD: 


Roy  Reiss,  Air  Force  Studies  &  Analyses  Agency 
COL  Hoa  Generazio,  ODUSA(OR) 

Mr.  Greg  Melcher,  N81 

Dr.  George  Akst,  MCCDC  Studies  &  Analysis 

COL  Wilmer  Sweetser,  JS  (J8) 

Mr.  Jim  Bexfield,  FS,  OSD(PA&E) 
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MORS  Workshop  Briefs: 


Analyzing  EBO  -  Dr.  Richard  Hayes  &  Sue  Iwanski 
OR  Methods  for  10  -  Dr.  Dick  Deckro 

Advancing  C4ISR  -  Dr.  Stuart  Starr,  Charles  Taylor  and  Cy  Staniec 
Better  Data  Practices  -  Tom  Allen 
Influence  C&C  -  Priscilla  Glasow 
Decision  Support  -  Pat  McKenna 

10.  Administrative  - 

Name  -  Operations  Analysis  Support  to  Network  Centric  Operations 

Dates  -  27-29  January  2004 

Location  -  Booz  Allen  Hamilton,  McLean,  VA 

Fees  - 

Mini-Symposium  Only:  US  Federal  Government  $115  and  $230  for  all  others 

Entire  Workshop:  US  Federal  Government  $225  and  $450  for  all  others 

Attendance  -  200  people,  by  invitation 
Classification  -  Unclassified 
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MORS  Special  Meeting  Planning  Committee 

MORS  Workshop: 

Operations  Analysis  Support  to  Network  Centric  Operations 


1.  Core  Group  - 
General  Co-Chairs: 

Technical  Co-Chairs: 

Synthesis  Chair: 

Group: 


Site  Coordinator: 


Dennis  Baer,  Northrop  Grumman  IT 
Kirk  Michealson,  Lockheed  Martin  ISS/AC-T 

Dr.  Richard  Hayes,  Evidence  Based  Research,  Inc. 
Dr.  David  Alberts,  OSD(Nn) 

Dr.  Jerry  Kotchka,  FS,  Lockheed  Martin  ISS/AC-T 
CAPT  Jeff  Kline,  NFS 

Michael  Bauman,  FS,  TRAC  Ft.  Leavenworth 
David  Garvey,  Alidade  Incorporated 
Dr.  Stuart  Starr,  FS,  The  MITRE  Corporation 
Dr.  Russ  Richards,  JFCOM  (MITRE) 

Dr.  Greg  Parnell,  FS,  USMA 

ProfBillKemple,NPS 

Bob  Manke,  Naval  Undersea  Warfare  Center 

Booz  Allen  Hamilton 


Administrative  Coordinators:  Brian  Engler,  Executive  Vice-President,  MORS 

Natalie  Kelly,  Vice-President  for  Admin,  MORS 

MORS  Bulldog:  Dean  Free,  Anteon 

2.  Working  Groups  - 

a.  WGl  -  Measures  of  Merit 

Chair  -  Jim  Jacobs,  Northrop  Grumman  IT 
Co-Chair  -  Corinne  Wallshein,  AFS  AA 

Advisor  -  Dr.  Kimberly  Holloman,  Evidence  Based  Research,  Inc. 

b.  WG  2  -  Processes  &  Methodologies: 

Chair  -  Dr.  Daniel  Maxwell,  Evidence  Based  Research,  Inc. 

Co-Chair  -  Ralph  Klingbeil,  Naval  Undersea  Warfare  Center 
Co-Chair  -  Jeffrey  Vick,  Boeing  Military  Studies  &  Analysis 
Advisor  -  Graham  Mathieson,  UK  Defence  Science  &  Tech  Lab 
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c.  WG  3  -  Assessment  Tools: 


Chair  -  Steven  Beres,  Evidence  Based  Research,  Inc. 

Co-Chair  -  Dr.  Mark  Youngren,  The  MITRE  Corporation 
Co-Chair  -  Ben  Taylor,  UK  Defence  Science  &  Tech  Lab 
Co-Chair  -  TJ  Ferrell,  Boeing  Military  Studies  &  Analyses 
Advisor  -  LtCol  Gregory  McIntyre,  JWARS,  OSD(PA&E) 

d.  WG  4  -  Force  Transformation  Process: 

Chair  -  Bob  Gregg,  Whitney,  Bradley  &  Brown,  Inc. 

Co-Chair  -  Duncan  Tailby,  JFCOM  -  Analysis  Division 
Advisor  -  Dr.  John  Hanley,  IDA 

e.  WG  5  -  NCO  &  Coalition  Forces: 

Chair  -  COL  Pat  Vye,  ODUS  A(OR) 

Co-Chair  -  Ray  Christian,  Naval  Undersea  Warfare  Center 
Co-Chair  -  Joe  Quartararo,  JFCOM  -  Analysis  Division 
Co-Chair  -  Alan  Cowdale,  UK  Defence  Science  &  Tech  Lab 
Advisor  -  Walt  Perry,  RAND 

f.  WG  6  -  Applying  NCO  to  Actual  Events: 

Chair  -  MAJ  Sean  Deller,  JFCOM,  J9,  Lessons  Learned 
Co-Chair  -  MAJ  Greg  Petrick,  Hanscom  AFB,  Analysis  Division 
Advisor  -  COL  Steven  Mains,  JFCOM,  J9,  Lessons  Learned 

3.  Other  Members  - 

a.  Previous  MORS  Workshops  - 

Analyzing  EBO  —  Dr.  Richard  Hayes,  EBR,  Inc. 

Ms.  Sue  Iwanski,  Northrop  Grumman  IT 

OR  Methods  for  10  -  Dr.  Richard  Deckro,  AFIT 

LtCol  Greg  McIntyre,  AFSAA 


b.  At-Large  Members  - 

Jeff  Manickas,  Naval  Undersea  Warfare  Command 
George  Cran,  UK  Defence  Science  &  Tech  Lab 
Danielle  Martin,  EBR,  Inc. 
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c.  MORS  Leadership  - 

Dr.  Steven  Pilnick,  MORS  VP(MO) 

Mr.  William  Dunn,  MORS  Special  Meetings  Chair 
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MORS  Special  Meeting  Presentations 

MORS  Workshop: 

Operations  Analysis  Support  to  Network  Centric  Operations 


Tuesday.  January  27^’’.  Mini-Svmposium  - 

a.  Keynote  Presentation  - 

(1)  Transforming  Defense  -  VADM  Arthur  Cebrowski,  USN,  Retired 

b.  Service  Presentations  - 

(1)  Operations  Analysis  Support  to  Network  Centric  Operations:  A  Joint  View  - 
COL  Steven  Mains,  Joint  Forces  Command,  J-9,  Lessons  Learned 

(2)  Operations  Analysis  Support  to  Network  Centric  Operations:  An  Army  Perspective  - 
Mr.  Michael  Bauman,  Director,  TRADOC  Analysis  Center,  Fort  Leavenworth,  KS 

(3)  Analysis  for  Network  Centric  Warfare  in  the  Navy  —  Dr.  Michael  Bell,  Senior 
Analyst,  FORCEnet  Requirements  Branch,  Office  of  the  CNO  (N61F) 

(4)  Operations  Analysis  Support  to  Network  Centric  Operations:  UK  Overview  -  Dr. 
George  Cran,  Defence  Science  and  Technology  Laboratory  (DSTL)  Network  Enabled 
Capability  (NEC)  Programme  Leader,  United  Kingdom’s  Ministry  of  Defence 

c.  Framework  Presentations  - 

(1)  in  the  Information  Age:  The  Last  Mile  of  Transformation  —  Developed  by  Dr. 
David  Alberts,  Director,  Research  and  Strategic  Planning,  OASD  (Nil);  presented  by 
Dr.  Richard  Hayes,  President,  Evidence  Based  Research,  Inc. 

(2)  Network  Centric  Operations:  Differences  in  Perspective  -  Dr.  Richard  Hayes, 
President,  Evidence  Based  Research,  Inc. 

d.  Guidance  Presentations  - 

(1)  Operations  Analysis  Process  -  Dr.  Jerry  Kotchka,  Senior  Operations  Analyst, 
Lockheed  Martin  Advanced  Concepts 

(2)  Summary  of  the  MORS  Workshop:  "Analyzing  Effects-Based  Operations  ”  -  Dr. 
Richard  Hayes,  President,  Evidence  Based  Research,  Inc.  and  Ms.  Sue  Iwanski, 

Senior  Operations  Analyst,  Northrop  Grumman  Information  Technology 

(3)  Summary  of  the  MORS  Workshop:  "Operations  Research  Methods  for  Information 

Operations  -  A  Battlespace  of  the  Century  ”  —  Developed  by  Dr.  Richard  Deckro, 

Professor,  Operations  Research  Department,  Air  Force  Institute  of  Technology; 
presented  by  LtCol  Greg  McIntyre,  Deputy  Director,  Analyses  Foundations,  Air 
Force  Studies  and  Analysis 
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(4)  Working  Group  Guidance  -  Mr.  Kirk  Michealson,  Senior  Operations  Analyst, 
Lockheed  Martin  Advanced  Concepts 

(5)  Technical  Chair  Guidance  -  Dr.  David  Alberts,  Director,  Research  and  Strategic 
Planning,  OASD  (NH)  and  Dr.  Richard  Hayes,  President,  Evidence  Based  Research, 
Inc. 

(6)  Synthesis  Group  Guidance  -  Dr.  Jerry  Kotchka,  Senior  Operations  Analyst,  Lockheed 
Martin  Advanced  Concepts 

2.  Wednesday.  January  28*^.  Workshop  Joint  Session  -  (Working  Groups  1, 2, 3  &  6) 

a.  Using  JWARS  for  Analysis  -  LtCol  Greg  McIntyre,  Deputy  Director,  Analyses 

Foundations,  AFSAA/SAA  and  Mr.  Donald  Bates,  Director,  JWARS  Office, 

OSD(PA&E) 

3.  Wednesdav-Thursdav.  January  28*’^-29*^.  Working  Group  Presentations  - 

a.  Working  Group  1 ,  Measures  of  Merit  - 

(1)  Using  JWARS  for  Analysis  -  LtCol  Greg  McIntyre,  Deputy  Director,  Analyses 
Foundations,  AFSAA/SAA  and  Mr.  Donald  Bates,  Director,  JWARS  Office, 
OSD(PA&E) 

(2)  NCO  Conceptual  Framework,  Dr.  Daniel  T.  Maxwell,  Evidence  Based  Research,  Inc. 

(3)  lERs,  the  Pk’s  of  Network  Modeling,  Mr.  Peter  M.  Kerekanich,  TRAC 

(4)  WIN -T  Analysis  of  Alternatives,  Emerging  Results,  Mr.  Steve  Herndon,  TRAC 

(5)  NC  Measures  of  Merit  from  a  Queueing  Theory  Perspective,  Dr.  Ralph  S.  Klingbeil, 
NUWCDIVWT 

(6)  Quantifying  Information  Availability  for  Situational  Awareness,  LCOL  Stephen  R. 
Riese,  TRAC 

b.  Working  Group  2,  Processes  and  Methodologies  - 

(1)  Using  JWARS  for  Analysis  -  LtCol  Greg  McIntyre,  Deputy  Director,  Analyses 
Foundations,  AFSAA/SAA  and  Mr.  Donald  Bates,  Director,  JWARS  Office, 
OSD(PA&E) 

(2)  NCO  Conceptual  Framework,  Dr.  Daniel  T.  Maxwell,  Evidence  Based  Research,  Inc. 

(3)  Full  Spectrum  Analysis,  Graham  Mathieson,  Defense  Science  and  Technology 
Laboratory,  UK  MoD 

(4)  Sweden 's  Roadmap  to  Network-Based  Defence  Transformation,  Dr.  Anne  Worm, 
Swedish  Defence  Research  Agency 

c.  Working  Group  3,  Assessment  Tools  - 

(1)  Using  JWARS  for  Analysis  -  LtCol  Greg  McIntyre,  Deputy  Director,  Analyses 
Foundations,  AFSAA/SAA  and  Mr.  Donald  Bates,  Director,  JWARS  Office, 
OSD(PA&E) 
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(2)  UK  High-level  Campaign  models  and  NEC  related  Studies,  Dr.  Ben  Taylor  and 
Lynda  Sharp,  Defense  Science  and  Technology  Laboratory,  UK  MoD 

(3)  CASTFOREM,  Doug  Mackey,  TRAC-WSMR 

(4)  Quantifying  Information  Availability  for  Situational  Awareness,  LCOL  Stephen  R. 
Riese,  TRAC 

Working  Group  4,  NCO  and  Force  Transformation  — 

(1)  Strategic  Planning  and  Force  Transformation,  Duncan  Tailby,  Defense  Science  and 
Technology  Organization,  Australia 

(2)  Joint  Prototype  Testing,  Joe  Drelling,  Joint  Forces  Command  J9/MYMIC  LLC 

Working  Group  5,  NCO  and  Coalition  Forces  - 

(1)  Insights  from  UK  Data  Link  and  Situational  Awareness  Studies  (UK) 

(2)  NetCentric  Operations  in  the  Canadian  Context  (CA) 

(3)  Network  Analysis  from  a  Swedish  OR  Perspective  (SW) 

(4)  Network  Centric  Operations  and  the  Joint  Interagency  Coordination  Group 
Workshop  Results  (US) 

(5)  Analysis  of  Carrier  Task  Force  Network  Centric  Warfare  Architectures  for  Network 
Centric  Defense  (US) 

(6)  An  Analytic  Approach  to  Network  Centric  Warfare  Metrics  and  their  Relationships: 
From  Connectivity  to  Operational  Tempo  (AU) 

(7)  A  Functional  Survey  of  Netted  Warfare  Analysis  Methods  and  Selected  Applications 
to  Coalition  Maritime  Tactical  Situations  (US) 

(8)  Dutch  and  German  Forces  Employment  of  Network-Centric  Capabilities  During 
Peace  Keeping  Operations  (US) 

(9)  ACIDC  Developments  From  Operation  Iraqi  Freedom  Lessons  Identified  (UK) 

(10)  Analysis  From  Multi-National  Limited  Objective  Experiment  2  (US) 

(11)  Analysis  Plan  for  Multi-National  Experiment  3  (US) 

Working  Group  6,  Appling  OA  Support  to  an  Actual  NCO  Event  - 

(1)  Using  JWARS  for  Analysis  -  LtCol  Greg  McIntyre,  Deputy  Director,  Analyses 
Foundations,  AFSAA/SAA  and  Mr.  Donald  Bates,  Director,  JWARS  Office, 
OSD(PA&E) 

(2)  Joint  Lessons  Learned:  Network  Centric  Warfare  Observations  in  Operation  IRAQI 
FREEDOM,  by  Maj  Michele  Cook,  Joint  Lessons  Learned  Team,  US  JFCOM 

(3)  Providing  Analytical  Support  to  SWA  Phase-IV  Ops  with  Representation  in  Theater 
(PASSPORT),  by  CPT  Allison  Stewart,  Center  for  Army  Analysis 
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Sponsors  Briefing 
27  February  2004 
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Military  Operations  Research  Society 


•  What  was  the  motivation  for  this 
meeting? 

•  How  it  was  organized? 

•  What  we  found  out? 

•  Where  do  we  go  from  here? 
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math&tkm  for  this  mms 


MUJtary  OperatJorw  Research  Society 


•  Requested  by  Jeff  Manickas,  NUWC  in  previous 
workshop 

•  Sponsors  inputs  of  Special  Meetings 

♦  #1  ranked  by  OSD  and  Navy 

♦  #2  ranked  by  Army 

♦  #2  overall  of  all  Sponsors 

•  Encourage  new  membership  and  coalition 
participation 

♦  152  analysts  attended,  of  which  58  were  new  to  MORS  or  non-members 

♦  26  foreign  personnel  from:  UK,  Canada,  Australia,  Netherlands,  Denmark, 
Sweden  and  Israel 


Jeff  Manickas  recommended  at  the  New  Analysis  Techniques  Tutorial 
Workshop  at  JHU/APL  in  November  2002. 

152  analysts  and  decision  makers  participated!!  Of  these,  94  were  existing  or 
former  members  and  58  were  new  to  MORS  or  non-members.  Included  in  the 
latter  number  were  26  foreign  personnel  from:  UK,  Canada,  Australia, 
Netherlands,  Denmark,  Sweden  and  Israel. 
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Hqw  it  was  organhedf 

Schedule 


Hintsry  Operations  Research  Society 


*  1st  Pay  .  Mini-symposium 

♦  Keynote  Address  -  VADM  Cebrowski,  USN,  (Ret.) 

♦  Service  and  Coalition  Presentations 

♦  Framework  Presentations 

♦  Guidance  Presentations 

•  2"^  and  day  -  Workshop 

♦  WG  1:  Measures  of  Merit 

♦  WG  2:  Processes  and  Methodologies 

♦  WG3:  Assessment  Tools 

♦  WG  4:  NCO  and  the  Force  Transformation  Process 

♦  WG  5:  NCO  and  Coalition  Forces 

♦  WG  6:  Applying  NCO  to  an  Actual  Event 

♦  Synthesis  Group 


Core  Leadership 

Co-Chairs:  Dennis  Baer,  Northrop  Grumman  IT  and  Kirk  Michealson, 
Lockheed  Martin  -  ISS/AC-T 

Technical  Chairs:  Dr.  Richard  Hayes,  Evidence  Based  Research,  Inc. 
and  Dr.  David  Alberts,  OSD(NII) 

WG  1  Chair:  Jim  Jacobs,  Northrop  Grumman  IT 

WG  2  Chair:  Dr.  Daniel  Maxwell,  Evidence  Based  Research,  Inc. 

WG  3  Chair:  Steven  Beres,  Evidence  Based  Research,  Inc. 

WG  4  Chair:  Bob  Gregg,  Whitney,  Bradley  &  Brown,  Inc. 

WG  5  Chair:  COL  Pat  Vye,  ODUSA(OR) 

WG  6  Chair:  Maj  Sean  Deller,  JFCOM  (Lessons  Learned) 

Synthesis  Group  Chair:  Dr.  Jerry  Kotchka,  FS,  Lockheed  Martin  -  ISS/AC-T 
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How  it  wm  organized? 

Objectives  of  Meeting 


MlUtary  Operations  Research  Society 


•  Bring  together  a  multi-disciplined  group  of 
analysts,  operators  and  engineers  from 
across  service  organizations  to  share  their 
work. 

•  Develop  a  common  view  of  the  state  of 
practice. 

•  Expose  members  of  the  broader  analytic 
community  to  their  needs. 

•  Identify  shortfalls  and  recommend  solutions 
for  improving  the  state  of  practice. 


As  Network  Centric  Operations  (NCO)  are  being  developed,  the  analytic 
community  is  being  called  upon  to  support  the  military  services  with 
assessments  of  needs  and  capability  gaps  to  support  decisions  concerning 
options  under  consideration  by  the  US  Armed  Forces  and  its  coalition 
partners.  Many  organizations  are  interested  and  involved  in  employing  and 
analyzing  Network  Centric  Operations.  With  the  emergence  of  "Information 
Age"  warfare,  the  Operations  Analysis  (OA)  community  recognizes  it  must  play 
a  leadership  role  in  creating  and  refining  needed  metrics,  processes, 
methodologies,  models  and  simulations.  The  community  acknowledges  an 
urgent  need  to  share  its  efforts,  successes  and  failures  in  developing  the  key 
capabilities  required  for  analytically  rigorous  assessments  of  NCO.  On  27-29 
January  2004,  MORS  held  a  workshop  on  Operations  Analysis  Support  to 
Network  Centric  Operations  at  the  Booz  Allen  Hamilton  Conference  Center  in 
McLean,  VA.  The  goal  of  the  meeting  was  to  bring  together  a  multi- 
disciplined  group  of  analysts,  operators  and  engineers  from  across  service 
organizations  to  share  their  work,  develop  a  common  view  of  the  state  of 
practice,  expose  members  of  the  broader  analytic  community  to  their 
needs,  identify  shortfalls  and  recommend  solutions  for  improving  the  state  of 
practice. 
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mms 

Military  Operabons  Research  Society 

Broad  Observations 


•  More  Senior  analytical  leaders  and 
experienced  analysts  in  MORS 
workshops. 

•  Joint  business  models 

•  Networking  of  analysts 

•  NCO  addresses  a  set  of  Measure  of 
Merits  (MOMs),  not  by  a  single  MOM 


A  few  broad  observations  of  the  workshop  were  noted.  First,  there  exists  a  need 
to  include  senior  service  analytical  leaders  and  experienced  analysts  in  MORS 
workshops  to  enhance  interactions,  findings,  recommendations  and  to  mentor 
more  junior  participants.  Second,  a  joint  business  model  is  needed  that  enables 
services  and  joint  commands  to  conduct  operations  analysis  in  support  of 
emerging  NCO  concepts.  This  will  facilitate  the  initiation  of  OA  activity  early 
in  the  planning  process.  Third,  networking  of  operations  analysis  centers  and 
operation  analysts  along  with  adequate  reach  backreach-back  should  provide  a 
significant  improvement  in  support  of  decision  makers.  However,  an 
environment  of  trust  is  needed  among  analysts  and  between  the  analyst  and 
decision  maker.  Finally,  that  NCO  analysis  will  not  be  addressed  by  a  single 
Measure  of  Merit  (MOM)  but  by  a  set  of  MOMs. 
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Keynote  Address 


•  Highlights  of  presentations  for  working  group 
consideration. 

♦  Speed  and  agility  should  be  achieved  over  optimization. 

♦  Warfighter  advantage  exploits  behavioral  change  and  new  doctrine 
to  enable  self-synchronization,  speed  of  command  and  increased 
combat  power. 

♦  Information  sharing  is  a  new  source  of  power. 

♦  Measuring  values,  attitudes  and  beliefs  should  be  considered. 

•  Issued  a  challenge  to  the  MORS  Community  - 
to  create  tools  that  could  be  used  for  NCO  and 
transformation. 


VADM  Arthur  Cebrowki,  Director  -  Force  Transformation,  provided  the 
keynote  address.  Some  of  the  highlights  from  his  presentation  the  participants 
could  consider  for  their  working  group  discussions  included:  1)  speed  and  agility 
should  be  achieved  over  optimization;  2)  the  warfighter  advantage  exploits 
behavioral  change  and  new  doctrine  to  enable  self-synchronization,  speed  of 
command  and  increased  combat  power;  3)  information  sharing  is  a  new  source 
of  power;  and,  4)  measuring  values,  attitudes  and  beliefs  should  be  considered. 
During  his  presentation,  he  also  issued  a  challenge  to  the  MORS  Commimity — 
to  create  tools  that  could  be  used  for  NCO  and  transfonnation. 
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Service  and  Coalition  Presentations 

•  COL  Steven  Mains  (JFCOM,  J9,  Deputy  Director, 

Lessons  Learned  Group)  provided  the  Joint  Staff 
perception. 

♦  Data  is  found  -  Information  is  made. 

♦  Collection  does  not  equal  knowledge.  A  huge  “frame 
rate”  could  be  counterproductive. 

♦  Networks  do  not  produce  knowledge.  They  pass  data 
between  people  that  can  turn  it  into  information  and 
information  between  people  that  can  turn  it  into 
knowledge. 

♦  Knowledge  is  the  key  to  Network-Centric  Operations. 
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MUItaty  Operations  Research  Society 


•  Mr.  Michael  Bauman,  FS,  Director,  TRADOC  Analysis 
Center,  presented  the  Army  point  of  view  highlighting 
the  analysis  completed  for  the  Future  Combat  System. 

♦  The  FCS  program  and  focused  Army  investments  in  new  modeling 
have  advanced  the  Army’s  ability  to  model  and  analyze  network- 
enabled  operations  (aka  NCO). 

♦  The  military  OA  community  still  faces  challenges  though 

->  Complexity  of  NCO  scenarios,  overwhelming  data  and  ambiguous  cause-and- 
effects. 


Metrics  useful  to  analysts  and  decision  makers  that  also  resonate  with 
warfighters. 

♦  The  “business  model”  to  enable  OA  of  joint  operations,  to  include 
NCO,  is  still  an  exploratory  work-in-progress. 

A  means  to  generate  data  for  joint  system  concepts  is  evolving. 

^  A  mechanism  to  achieve  authoritative  representation  of  future  joint  concepts  is 


much  less  mature. 
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Service  and  Coalition  Presentations  (con*t) 


•  Dr.  Michael  Bell,  Senior  Analyst,  FORCEnet 
Requirements,  N61,  followed  with  the  Navy’s 
status  with  their  NCO  system,  FORCEnet. 
Some  highlights  included: 

♦Demonstrated  FORCEnet’s  value  of  analytical 
framework:. 

"^Connects  FORCEnet  capabilities  and  NCW  Framework 
•♦Capabilities  under  revision 

•♦Quantitative  measures  partially  successful  and  improvements 
proposed 

♦Improved  representation  of  how  knowledge  is 
achieved  in  C4ISR  campaign  analysis. 
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Service  and  Coalition  Presentations  (conH) 

•  Dr.  George  Cran,  UK  MoD  Network  Enabled  Capability  (NEC) 
Programme  Leader,  presented  the  United  Kingdom  view.  Dr.  Cran 
highlighted  the  operations  analysis  issues  for  NEC 

♦  Bringing  together  future  military  thinking  with  future  technologicai 
capabiiities  across  the  spectrum  of  operations  and  levels  of 
command. 

♦  Thinking  out  of  the  box  -  doing  better  things. 

♦  Bringing  all  Lines  of  Development  into  the  analysis,  especially 
people  issues. 

♦  Bringing  practicalities  to  the  vision:  drawing  operational  lessons 
learned  and  experimentation  into  the  analysis. 

♦  Enhancing  the  tool-set  to  reflect  NEC  with  greater  fidelity. 

♦  Conducting  Balance  of  Investment  studies  between  ‘hard’  systems 
and  information  systems. 
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Framework  Presentations 
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The  Technical  Co-  Chairs,  Dr. 
David  Alberts  and  Dr. 

Richard  Hayes,  provided  two 
framework  discussions: 

♦  “02  in  the  Information  Age: 
The  Last  Mile  of 
Transformation” 

♦“Network  Centric 
Operations:  Differences  in 
Perspective” 


NCO  Conceptual  Framework 
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The  first,  titled  “C2  in  the  Information  Age:  The  Last  Mile  of  Transformation,” 
was  intended  to  provide  a  common  definition  of  Network  Centric  Operations  for 
the  working  groups.  Some  of  the  discussion  points  included  the  value  of  the 
NCO  Framework,  the  prerequisites  for  transformation,  power  to  the  edge, 
making  it  happen  -  becoming  an  “edge”  organization,  and  the  OSD(NII) 
initiatives  for  building,  populating  and  protecting  the  network.  The  second, 
titled  “Network  Centric  Operations:  Differences  in  Perspective,”  was  offered  to 
present  the  Network  Centric  differences  among  the  United  States,  United 
Kingdom,  Sweden,  Australia  and  NATO. 


Appendix  E  - 12 


Framework  Presentations  (con*t) 


Military  Operation;  Research  Society 


•  As  one  of  the  Technical  Co-Chairs  for  the  Analyzing  Effects-Based 
Operations  Workshop,  Ms.  Sue  Iwanski  passed  on  the  following  take¬ 
away  thoughts: 

♦The  term  “Indicators  of  Success"  was  used  for  EBO  since  it  expands  beyond  traditional  Measures,  NCO  may 
also  require  non-traditional  Measures. 

♦  Fundamental  Sciences  were  considered  to  be  useful  for  EBO. 

♦The  EBO  tool  chest  was  envisioned  to  include  easily  manipulated,  specialized  modeling  and  simulation  tools, 
computational  social  science  tools,  data  mining,  colored  Petri  nets,  neural  networks,  and  specialized  tools 
developed  In  particular  application  areas. 

♦Networks  are  targets  in  EBO.  Information  exchange  Is  vital  to  NCO. 

♦  As  the  Chair  for  the  Operations  Research  Methods  for  Information 
Operations  Workshop,  Dr.  Richard  Deckro  prepared  a  list  of  points  to 
consider  for  the  participants: 

♦Build  MOE’s  and  Battle  Damage  Assessments  for  NCO 
♦Extend  into  EBO  MOEs  for  NCO  use 
♦What  should  a  NCO  “JMEMS"  look  like? 

♦Does  the  NCO  framework  help? 

♦What  do  Commanders  need  to  use  NCO  as  a  "primary  arrow"  in  their  quiver? 

♦Leadership  understanding 
♦Tools 

♦  Intel 

♦What  are  the  human  element  needs? 
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Guidance  Presentations 


•  The  working  groups  were  asked,  if  possible,  to  provide  assessments  to 
revitalize  the  state  of  analytical  practice,  i.e.,  their  “State  of  Health”  in 
two  areas;  the  Operations  Analysis  Process  and  the  Network  Centric 
Warfare  four  key  domains. 


Operations  Analysis  Process 


New  Four  Key  Domains 


Physical  Domain 

where  strike,  protect,  and  maneuver  take  place  across 
_ different  environments  .  , 

Information  Domaita  , 

:  where  information  is  created,  manipulated  and  shared  v 

y,,,  ,  Cognitive  Domain! 

•  where  perc^tions,  awareness,  beliefs,  ahei  values  reside  - 
and  where,  as  a  resiilt  of  sensemkinfe.  decisions  are  made 

Social  Domain 

where  force  entitieslnteract  J,/ 
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Final  assessment  by  Synthesis  Group 

Operations  Analysis  Process  NCW  Four  Key  Domains 
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Information  Domain 

nil  where  information  is  created,  manipulated  and  shared 

Cognitive  D  om  sdn 

where  perceptions,  awareness,  beliefs,  and  values  reside 
\  and  where,  as  a  result  of  sensemaking,  decisions  are  made 


Network  Centric  Warfare  Operations  can  be  represented  using  four  key  domains 
(physical,  information,  cognitive  and  social),  the  state  of  health  for  OA,  in 
relation  to  these  domains,  was  estimated  as  deficient  or  poor.  Even  at  the 
physical  level  of  modeling  and  analysis,  it  appears  that  the  Operations  Analysis 
community  continues  to  focus  on  the  warfare  (combat)  portion  of  NCO  and  not 
on  the  larger  set  of  military  missions  (e.g.,  OOTW)  -  as  evidenced  by  the 
scarcity  of  methods  for  these  situations.  In  general,  applying  OA  across  the  four 
domains  was  considered  fair  or  poor:  physical  -  fair,  information  -  fair, 
cognitive  -  poor,  and  social  -  poor.  Overall,  the  OA  community  seems  to  be 
uncomfortable  with  expanding  their  analysis  into  the  cognitive  and  social  levels. 
Scientific  approaches  of  the  past  that  addressed  the  physical  domain  well 
apparently  do  not  represent  the  other  domains  well,  and  are  usually  limited  in 
focus  to  key  physical  interactions  associated  with  combat  and  combat  support 
operations.  Methods  fi-om  other  disciplines,  such  as  the  medical  medicine, 
genetics,  sociology,  psychology,  political  science,  and  cultural  anthropology 
fields,  should  be  considered  as  potential  sources  of  knowledge  that  could  inform 
the  OA  process  and  community. 
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WG 1:  Measures  of  Merit 


Hllttary  Operations  Research  Society 


•  The  findings  focused  on  the  assessment  that  the  MOMs 
.  for  NCO,  especially  for  understanding  a  networked 
“cause  and  effect”  relationship,  are  a  work-in-progress. 
However,  previous  MOMs  for  the  physical  domains  for 
warfighting  missions  are  well  understood  and  remain 
relevant. 

♦  MOMs  must  both  enable  the  analyst  and  inform  the  client;  these  may  be 
starkly  different  for  Network  Centric  Operations  Analysis  (NCOA).  MOMs 
for  NCOA  are  a  work-in-progress. 

♦  Survivability-related  MOMs  are  increasingly  relevant  to  military  operations 
and  remain  so  for  NCOA  as  well  (at  all  levels,  i.e.,  MOPs,  MOEs). 

♦  A  combination  of  MOMs  is  necessary  for  OA  of  networked  “cause-and- 
effecr  relationships. 
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WG  2:  Processes  md  Methodologies 


Military  Operations  Research  Sod^ 


•  The  findings  focused  on  the  NCW  conceptual 
framework  and  the  gap  between  sense-making  and 
decision  making.  It  was  also  suggested  that  the 
relationship  between  cognitive  modeling  and  sense¬ 
making  modeling  needs  to  be  addressed. 

♦  The  NCO  framework  emphasizes  “Quality  of  Individual  Sense-making”  and 
the  “Quality  of  Group  Sense-making”  versus  decision  making. 

♦  Many  In  the  NCO  community  assert  that  cognitive  modeling  is  an  essential 
part  of  the  solution  to  modeling  sense-making. 

♦  The  lack  of  processes  and  methodologies  for  OA  using  sense-making  and 
cognitive  modeling  make  the  NCO  framework  a  challenge  to 
operationalize. 
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WG3:  Assessment  Tools 


•  The  findings  were  along  three  dimensions:  1) 
“better”  data;  2)  improvement  in  NCO 
assessment  tools;  3)  and,  the  netting  of 
assessment  tools  in  order  to  achieve  synergy 
in  the  application  of  a  tool  set. 

♦  Some  tools  exist,  but  the  data  domain  (both  real  blue  and  red  data) 
Is  still  too  stove-piped. 

♦  The  methods  to  change  information  to  knowledge  seem  to  exist 
within  the  models,  but  are  not  well  understood. 

♦  Netting  assessment  tools  is  necessary  but  not  sufficient;  we  must 
learn  to  “do  better  things,"  not  “do  things  better.” 

♦  The  networked  force  must  be  treated  as  a  holistic  entity  not  as  a 
sum  of  discrete  elements. 
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WG  4:  NCO  and  the  Farce  Transformatian  Process 

•  To  effect  Force  Transformation  it  was 
concluded  that  the  further  we  move  away 
from  assessing  transformations  that  are 
focused  on  a  single  Service,  tactics  and 
material  (“old”  missions,  etc.),  the  less 
OA  tools  are  available  to  the  analyst  for 
information,  cognitive  and  social  domain 
related  assessments. 
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WG  5:  NCO  and  Coalition  Forces 

•  It  was  discussed  that  the  OA  timelines  are  not  fast 
enough  to  keep  pace  with  technology  and  other 
developments.  The  experience  of  the  UK’s  embedded 
OAs  needs  to  be  assessed  and  possibly  followed. 
However,  the  inclusion  of  civilian  agencies,  such  as  the 
Department  of  Homeland  Security,  must  also  be 
addressed. 

♦  OA  analysis  timelines  are  not  fast  enough  to  meet  the  pace  of  technology 
and  DOTMLPF  development. 

♦  OA  is  overly  focused  on  military  to  the  exclusion  of  civiiian  agencies. 

♦  Need  to  consider  new  level  of  embedded  analysis  (OA  in-theater  with  data 
mining  and  assessment  tools)  to  provide  real  time  feedback  on  NCO 
effectiveness. 
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•  It  was  found  that  because  policies  and  doctrine  to 
incorporate  anaiysis  is  iacking,  along  with  the  minimal 
use  of  embedded  analysts,  that  OA  becomes  an 
afterthought.  If  these  findings  and  the  data  domain  are 
improved,  then  the  assessment  of  actual  NCO  events 
will  be  improved. 

♦  Lack  of  joint  and  some  Service  doctrine  to  Incorporate  analysis  in  a  JTF 
limits  OA  support  of  NCO. 

♦  Reliance  on  non-embedded  analysts  makes  OA  an  afterthought  in  JTFs. 

♦  OA  currently  limited  by  available  and  valid  data. 

♦  Time  and  data  constraints  limit  the  tool  set. 
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•  Among  the  Services 

♦  Two  dimensions:  Between  operators  and  between  analysts 

♦  Inconsistent  and  very  infrequent 

•  With  the  US  Agencies 

♦  Difficult  to  asses 

♦  Examples  unknown 

•  With  the  Coalition  Forces 

♦  Infrequent  and  inconsistent 

♦  Most  collaboration  is  on  a  bilateral  basis 


As  with  the  State  of  the  Practice,  each  of  the  working  groups  were  asked  to  provide  a  self-assessment  of 
the  level  of  collaboration  among  the  Services,  Coalition  Forces  and  US  Agencies  where  operations 
analysis  has  been  applied  to  NCO. 

The  level  of  collaboration  among  (and  within)  the  Services  was  assessed  as  inconsistent.  Collaboration 
between  services  is  varied  and  has  two  dimensions:  between  operators  and  between  analysts. 
Collaboration  between  analysts  was  good  with  the  Joint  Lessons  Learned  Team  because  they  were 
located  at  all  Combatant  and  Component  Headquarters  -  so  contact  was  physical.  Collaboration  between 
Service  teams  at  disparate  sites  was  very  infrequent.  Most  were  completely  unaware  of  the  others’ 
existence. 

The  collaboration  with  the  US  Agencies  was  difficult  to  assess  because  the  majority  of  the  attendees  did 
not  know  examples.  Working  Group  6  discussed  that  the  0/F  Phase  IV  planning  involved  almost  every 
major  agency  in  the  US  government;  however,  none  of  them  had  the  collaborative  capabilities  and 
expertise  to  contribute  effectively  and  in  a  timely  manner. 

For  the  Coalition  Forces,  collaboration  was  considered  infrequent  and  inconsistent.  A  spectrum  of 
change  of  military  forces  in  the  international  community  exists  from  status  quo  to  modernization  through 
transformation  offerees.  Nations  are  at  different  points  in  the  spectrum  with  respect  to  NCO  but  they  are 
committed  to  moving  away  from  the  status  quo  end  of  the  spectrum.  The  participation  to  various 
multinational  forums  is  evidence  of  the  commitment.  It  was  noted  that  most  collaboration  with  Coalition 
forces  is  on  a  bilateral  basis.  Many  of  the  activities  are  modest  events  between  coalition  partners  (entropy 
work,  agent  based  modeling,  S&T  assessment,  an  assessment  of  decision  making  in  collaborative 
environments,  development  of  a  conceptual  model,  and  working  groups  to  discuss  interoperability).  The 
conclusion  is  that  there  is  still  a  need  for  significant  collaborative  analytical  events  between  Coalition 
partners. 
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Synthesis  Group 


•  First,  the  Synthesis  Group  developed  a  set  of  actionable 
recommendations  from  the  working  group  discussions 
and  their  findings. 

♦  A  MORS  Symposium  working  group  or  composite  group 
dedicated  to  NCO  is  needed. 


♦  Create  a  center(s)  of  excellence  within  services  and 
JFCOM  for  NCO  assessments  to  capture  and  share 
results  and  data  of  experiments,  training,  analysis  and 
experience. 


♦  Enable  the  networking  of  analysts. 

♦  Establish  a  “business  model.” 

♦  Conduct  follow-on  MORS  special  meetings  to  share 
Network  Centric  Operations  analysis  and  experimentation 


First,  the  Synthesis  Group  developed  a  set  of  actionable  recommendations  from  the  working  group  discussions 
and  their  findings. 

•  A  MORS  Symposium  working  group  or  composite  group  dedicated  to  NCO  is  needed.  NCO  is  much  more 
than  C4ISR  and  Information  Operations!  This  new  working  group/composite  group  could  be  charged  to 
examine  the  OA  toolkit  and  processes  in  order  to  better  focus  on  the  NCO  problem.  At  the  72"'*  MORSS, 
propose  the  CG  B  Chair  discuss  this  issue  with  the  leadership  from  Working  Groups  5-10,  providing 
recommendations  to  the  MORS  Working  Group/Composite  Group  Committee  Chair  and  the  Vice  President 
(Meeting  Operations). 

•  Create  a  center(s)  of  excellence  within  services  and  JFCOM  for  NCO  assessments  to  capture  and  share 
results  and  data  of  experiments,  training,  analysis  and  experience. 

•  Enable  (thru  agreements,  business  practices)  the  networking  of  analysts  working  NCO  and  provide  reach- 
back  for  tools,  data,  and  previous  studies. 

•  Establish  a  “business  model”  that  enables  services  to  conduct  operations  analysis  in  support  of  “bom  joint” 
NCO  concepts  that  is  underpinned  by  a  set  of  approved  tools  and  data. 

•  Conduct  follow-on  MORS  special  meetings  to  share  Network  Centric  Operations  analysis  and 
experimentation  results.  There  were  two  options  recommended: 

A  2-day  or  3-day  Mini-Symposium  to  educate  the  community  in  the  area  of  NCO  analyses.  Training  and 
education  was  seen  to  be  critically  important  to  transformation  overall,  and  specifically,  to  the  operations 
analysis  practice  in  support  of  NCO. 

A  3-day  Workshop  to  discuss  the  issues  more  fully.  It  was  suggested  to  provide  foundation  presentations  during 
the  morning  of  the  1®‘  day  and  outbriefs  during  the  afternoon  of  the  3"**  day — ^this  would  leave  two  full  days  for 
discussion  and  further  debate.  During  this  workshop  the  working  groups  felt  they  only  “scratched”  the  surface 
in  their  discussions. 
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•  OA  process  developed  in  the  Industrial  Age 
has  applicability  to  Network  Centric  Operations 
(NCO)  in  the  information  Age. 

•  We  do  not  do  a  satisfactory  job  in 
documenting,  sharing  and  learning  from  NCO 
analyses  performed. 

•  With  the  arrival  of  terrorism,  other  asymmetric 
threats,  rogue  nations,  and  homeland  defense, 
NCO  is  an  opportunity  for  transformation. 
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Mlittarv  Operations  Research  Society 

Future  Challenges 

•  Metrics  developed  for  social  and  cognitive  domains. 

•  Tools  developed  or  adapted  to  changing  and  emerging 
concepts. 

•  Anticipating  next  transformational  capability  or 
technology  on  the  horizon. 

•  Coalition  warfare  analysis  is  lacking  in  today's 
assessment  tools. 

•  The  warfighting  culture  needs  to  be  changed  to  accept 
more  analysis,  i.e.,  emphasis  on  collecting  data  should 
start  in  the  planning  phase. 


The  working  group  attendees  were  asked  to  assess  their  future  expected  challenges. 

The  primary  challenge  for  measures  of  merit  is  to  keep  up  with  the  rapidly  changing 
scope  of  NCO.  Agility  is  a  desirable  feature  (for  which  measures  are  needed)  in  NCO. 
The  metrics  set  should  be  agile  as  well.  Not  only  do  we  need  more  and  better  measures 
for  NCO,  we  need  to  rigorously  address  the  issue  of  sufficiency.  How  much  is  enough? 
The  answer  may  differ  from  one  situation  to  the  next.  Development  of  standards  for  NCO 
analysis,  including  metrics,  faces  the  challenge  of  incorporating  the  terminology  and 
practices  of  the  social  sciences  in  order  to  establish  measures  in  the  social  and  cognitive 
domains. 

In  order  to  accurately  represent  NCO,  a  new  set  of  models  or  a  new  way  of  interoperating 
between  them  needs  to  be  created.  Today's  models  cannot  represent  NCO.  In  order  to 
accurately  represent  NCO  and  conduct  accurate  NCO  analysis,  all  aspects  of  NCO  need  to 
be  represented.  This  means  networking  not  only  two  battalions  but  also  all  actors  involved 
in  the  operation  including  reach-back  capabilities.  This  is  the  essence  of  NCO,  and  the 
capabilities  cannot  be  explored  if  they  are  not  represented.  Essentially,  the  tools  that  need 
to  be  developed  need  to  be  able  to  adapt  to  changing  and  emerging  concepts,  fri  a 
complex,  network  centric  world  the  analysis  challenges  will  continue  to  evolve.  We 
should  not  adopt  a  mindset  of  getting  the  tools  and  processes  “right” — ^we  need  an  ethos 
of  continuously  examining  our  tools  and  processes  to  make  them  better. 
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What  is  the  next  transformational  capability  or  technology  on  the  horizon? 
Analysts  can  and  should  be  engaged  in  this  effort,  and  clearly  must  look  beyond 
the  bounds  of  defense  industries  for  promising  opportunities.  By  taking  a  broad 
look  at  the  8  “key  elements”  of  transformation  (and  beyond)  we  may  uncover 
areas  for  further  investigation.  This  is  not  a  one-player  game — we  must  also 
think  about  areas  that  potential  adversaries  can  exploit  and  work  them  in  to  our 
future  scenarios  and  operating  concepts.  Palm  pilots  on  the  battlefield  are  an 
example  of  bottom-up  approaches  that  have  been  attempted  by  tactical  and 
operational  forces.  These  typically  are  not  officially  sanctioned.  Some  may 
have  great  value,  some  may  have  minimal  impact,  and  some  may  be  downright 
dangerous.  Analysts  need  to  engage  in  the  effort  to  assess  these  “bottom-up” 
initiatives. 

Coalition  warfare  analysis  is  lacking  in  today's  assessment  tools.  In  the  United 
States,  Joint  is  making  some  headway,  but  most  operations  in  the  future  will 
compose  Coalition  forces.  Throughout  recent  history,  the  ability  to  share 
information  in  a  Coalition  environment  is  a  great  difficulty  though  necessary. 
Other  issues  that  need  to  be  addressed  in  the  Coalition  environment  are 
linguistic  issues,  and  the  systems  interoperability.  NCO  most  certainly  is  about 
fostering  collaboration  and  networking  to  a  wide  degree.  We  must  expand  our 
network  of  analysts  in  include  interagency.  Coalition,  and  non-governmental 
partners. 

On  the  battlefield,  if  we  don’t  do  anything  the  information-sharing  problem  will 
actually  get  worse  because  the  planning,  execution  and  analysis  processes  are 
conducted  increasingly  on  the  SIPRNET.  This  has  the  effect  of  excluding  our 
Coalition  partners.  Over-classification  of  planning,  deployment,  and  execution 
and  sustainment  products  exacerbates  the  information-sharing  problem.  This 
adds  work  to  the  declassification  work  that  the  staff  Foreign  Disclosure  Officers 
must  do  prior  to  transferring  products  to  coalition  info  sharing  systems  like 
CENTRIX. 

The  warfighting  culture  needs  to  be  changed  to  accept  more  analysis.  This  will 
require  changes  in  doctrine  and  training  and  experience  of  leaders.  As  analysts 
do  more  collection  and  analysis  in  the  field,  the  procedures  for  data  collection 
will  mature,  but  emphasis  on  collecting  data  should  start  in  the  plaiming  phase. 
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HBJtanr  Operations  Research  Society 


•  Where  we  go  from  here? 

♦  DoD  Analysis  Community 

♦  MORS  Community 
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Where  we  go  from  here? 

DoD  Analysis  Community 


MiilUry  OperotiotM  Research  Society 


•  Synthesis  Group 

♦  Center  of  Excellence 

♦  Networking  of  Analysts 

♦  Establish  a  Business  Model 

•  Future  Challenges 

♦  Metrics 

♦  Tools 

♦  Technology  Assessment 

♦  Coalition  Warfare  Analysis 

♦  Integrating  Analysis  Into  Warfighting 
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Where  we  go  from  here? 

MORS  Community 

Military  Operation*  Research  Society 

•  Synthesis  Group 

•  Future  Challenges 

•  Composite  Group  for  NCO 

•  Follow-on  Meeting 
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MORS  Workshop: 
Decision  Aids/Support  to 
Joint  Operations  Planning 

Pat  McKenna,  USSTRATCOM,  McKennaP@stratcom.mil 

At  the  request  of  United  States  Strategic  Command 
(USSTRATCOM),  MORS  conducted  a  Workshop  on 
Decision  Aids/Support  to  Joint  Operations  Planning  at 
Offutt  AFB,  NE,  18-20  November  2003.  The  Air  Force  Studies 
and  Analyses  Agency  (AFSAA),  as  a  MORS  Sponsor,  was  a  pro¬ 
ponent  of  the  meeting.  Eighty-two  (82)  people  representing  all  the 
services,  the  Combatant  Commands,  OSD,  the  Joint  Staff,  and 
other  agencies  attended  the  conference.  Two  of  the  attendees  were 
from  the  United  Kingdom  (UK),  offering  a  valuable  perspective 
on  the  topic.  Forty  of  the  participants  were  attending  their  first 
MORS  event.  Pat  McKenna  and  Dr  Roy  Rice,  FS,  were  co¬ 
chairs  of  the  Workshop. 

The  focus  of  this  workshop  was  on  Military  Planning,  Joint 
Operation  Planning  (Crisis  and  Deliberate),  Joint  Operation  Plan¬ 
ning  and  Execution  System  (JOPES)  functions  of  Strategy  Deter¬ 
mination  and  Coui*se  of  Action  Development  The  intended  audi¬ 
ence  was  analysts  and  operational  planners  at  all  levels  of  military 
organizations.  Our  goals  and  objectives  were  to  identify  analytic 
approaches  that  might  be  used  to  enhance  the  JOPES  planning 
functions  of  Strategy  Detennination  and  Course  of  Action  Devel¬ 
opment. 

Specific  Objectives: 

•  Examine  techniques  of  eliciting  infomiation  from  decision  mak¬ 
ers  and  displaying  infoimation  back  to  decision  makers. 

•  Examine  the  implications  of  time  on  the  level  of  detailed  analy¬ 
sis  possible  and  how  tools/techniques  can  address  time/detail 
scaling  issues. 

•  Examine  tools/techniques/theories  that  could  be  applied  to  the 
JOPES  functions  of  Strategy  Determination  and  Course  of 
Action  Development. 

Background  on  JOPES 

As  discussed  in  Joint  Pub  5-0,  Doctrine  for  Planning  Joint 

(See  DECISION,/?.  36) 


MORS  Workshop: 
Operations  Analysis 
Support  to  Network 
Centric  Operations 

Kirk  Michealson,  Lockheed  Martin  Advanced  Concepts, 
kirk.a.michealson@Imco.com 

Dennis  Baer,  Northrop  Grumman  IT,  dennis.baer@ngc.com 

Background 

As  Network  Centric  Operations  (NCO)  are  being 
developed,  the  analytic  community  is  being  called  upon  to 
support  the  military  services  with  assessments  of  needs  and 
capability  gaps  to  support  decisions  concerning  options  under  con¬ 
sideration  by  the  US  Armed  Forces  and  its  coalition  partners. 
Many  organizations  are  interested  and  involved  in  employing  and 
analyzing  Network  Centric  Operations.  With  the  emergence  of 
“Information  Age”  warfare,  the  Operations  Analysis  (OA)  commu¬ 
nity  recognizes  it  must  play  a  leadership  role  in  creating  and  refin¬ 
ing  needed  metrics,  processes,  methodologies,  models  and  simula¬ 
tions.  The  community  also  acknowledges  an  urgent  need  to  share 
its  efforts,  successes  and  failures  in  developing  the  key  capabilities 
required  for  analytically  rigorous  assessments  of  NCO. 

On  27-29  Januaiy  2004,  MORS  held  a  workshop  on  Operations 
Atmlysis  Support  to  Network  Centric  Operations  at  the  Booz  Allen 
Hamilton  Conference  Center  in  McLean,  VA.  The  goal  of  the 
meeting  was  to  bring  together  a  multi-disciplined  group  of  analysts, 
operators  and  engineers  from  across  service  organizations  to  share 
their  work;  develop  a  common  view  of  the  state  of  practice;  expose 
members  of  the  broader  analytic  community  to  their  needs;  identify 
shortfalls  and  recommend  solutions  for  improving  the  state  of  prac¬ 
tice.  152  analysts  and  decision  makers  participated.  Of  these,  94 
were  current  or  former  members  and  58  were  new  to  MORS. 
Among  the  participants  were  26  foreign  personnel  from  the  UK, 
Canada,  Australia,  Netherlands,  Denmark,  Sweden  and  Israel. 

Workshop  Description 

Mini-Symposium  Overview  ( 1st  Day)  - 

The  mini-symposium  on  the  first  day  featured  presentations  to 
bring  us  up  to  speed  on  the  state  of  the  practice  of  operations 

(See  NCO,  - 
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(continued  from  p.  1) 

analysis  support  to  NCO.  After  the  wel¬ 
come  and  introductions  by  the  MORS 
President  (LTC  WiUie  McFadden,  USA), 
Host  (Mr  Steve  Stamer,  BAH)  and  Gen¬ 
eral  Co-Chairs  (Mr  Dennis  Baer, 
Northrop  Grumman  IT  and  Mr  Kirk 
Michealson,  Lockheed  Martin  Advanced 
Concepts),  VADM  Arthur  Cebrowski, 
USN  (Ret.),  Director,  Office  of  Force 
Transformation  provided  the  keynote 
address  on  “Transforming  Defense  and 
Implementing  Network  Centric  Warfare.” 

Next  the  Services  were  invited  to  pro¬ 
vide,  from  their  perspective,  an  overview 
of  the  challenges  and  progress  to  date 
using  operations  analysis  to  support  NCO, 
including  highlighting  context,  issues  and 
what  is  needed  from  tlie  analytical  com¬ 
munity,  and  identifying  some  of  the  broad 
analytical  and  difficult  questions  they 
have  observed.  The  Service  presentations 
were  provided  by  COL  Steven  Mains 
(Joint  Forces  Command,  J9,  Lessons 
Learned  Group);  Mr  Michael  Bauman, 
FS  (Director  TRADOC  Analysis  Center, 
Fort  Leavenworth,  KS);  Dr  Michael  Bell 
(Senior  Analyst,  FORCEnet  Require¬ 
ments  Branch,  Office  of  the  CNO);  and, 
Dr  George  Cran  (NEC  Programme 
Leader,  Defence  Science  and  Technology 
Laboratory,  UK  Ministry  of  Defence). 

Dr  Richard  Hayes,  President  of  Evi¬ 
dence  Based  Research,  Inc.  and  Work¬ 
shop  Technical  Co-Chair,  provided  two 
framework  presentations.  The  first 
offered  a  foundation  of  Network  Centric 
Operations  in  a  brief  titled  “C2  in  the 
Infonnation  Age:  The  Last  Mile  of  Trans¬ 
formation,”  while  the  second  discussed 
the  conceptual  differences  between  the 
United  States  and  Coalition  Countries 
with  respect  to  NCO  and  Transformation. 

The  final  sessions  of  the  day  were  pre¬ 
sentations  that  offered  guidance  for  the 
remaining  two  days  of  the  workshop.  Dr 
Jerry  Kotchka,  FS  (Lockheed  Martin 
Advanced  Concepts)  provided  a  review  of 
the  operations  analysis  process;  Ms  Sue 
Iwanksi  (Northrop  Grumman  IT)  fol¬ 
lowed  with  an  overview  of  the  MORS 
Analyzing  Effects-Based  Operations 
Workshop;  and,  LtCol  Greg  McIntyre 
(Air  Force  Studies  and  Analyses  Agency) 
offered  an  overview  of  the  MORS  Opera¬ 
tions  Research  Methods  for  Infomiation 
Operations  Workshop. 


Workshop  Overview  (2nd  <Sc  3rd  Days)  - 

The  mini-symposium  was  followed  by 
a  two-day  workshop.  The  participants 
met  in  six  working  groups:  (1)  Measures 
of  Merit;  (2)  Processes  and  Methodolo¬ 
gies;  (3)  Assessment  Tools;  (4)  NCO  cmd 
Force  Transformation;  (5)  NCO  and 
Coalition  Forces;  and,  (6)  Applying  NCO 
to  Actual  Events,  A  Synthesis  Group  was 
formed  to  examine  insights  across  all  six 
working  groups  and  to  help  bring  together 
a  consistent  set  of  workshop  findings. 

NCO  Definition 

A  common  definition  of  Network  Cen¬ 
tric  Operations  was  developed  by  the 
Technical  Co-Chairs  (Dr  David  Alberts, 
OSD  (NH),  and  Dr  Richard  Hayes).  Net¬ 
work  Centric  Operations  involves  the 
development  and  employment  of  mission 
capability  packages  that  are  the  embodi¬ 
ment  of  the  tenets  of  Network  Centric 
Warfare  (NCW)  in  operations  across  the 
full  mission  spectrum.  These  tenets  state 
that  a  robustly  networked  force  improves 
information  sharing  and  collaboration, 
which  enhances  the  quality  of  informa¬ 
tion,  the  quality  of  awareness,  and 
improves  shared  situational  awareness. 
This  results  in  enhanced  collaboration  and 
enables  self-synchronization,  improving 
sustainability  and  increasing  the  speed  of 
command,  all  of  which  ultimately  result  in 
dramatically  increased  mission  effective¬ 
ness. 

The  principles  of  Effects  Based  Opera¬ 
tions  (EBO)  go  hand-in-hand  with  NCO 
because  warfare,  particularly  effective 
warfare,  has  always  been  effects-based. 
Sun  Tzu,  Genghis  Khan,  Napoleon, 
Eisenhower  and  Schwarzkopf  all  would 
be  familiar  with  the  principles  that  (1) 
warfare  should  include  all  the  instruments 
of  national  power  and  that  (2)  each  instru¬ 
ment  should  be  applied  in  a  way  that  max¬ 
imizes  its  desirable  impact,  minimizes 
undesirable  ones,  and  complements 
actions  taken  in  other  arenas.  These  basic 
principles,  which  define  the  essence  of 
EBO,  occur  in  a  context  that  makes  them 
particularly  relevant  today.  First,  we  have 
the  means  to  gather,  integrate,  and  apply 
more  data,  information,  and  knowledge 
than  analysts  and  policy  makers  in  earlier 
eras  —  we  are  in  the  Infonnation  Age. 
The  tenets  of  NCW  address  these  means 
and  postulate  how  they  can  increase  mis¬ 
sion  effectiveness.  The  seven  tenets  of 


NCW  are: 

1.  Robustly  networked  force 

2.  Information  sharing 

3.  Collaboration 

4.  Quality  of  infonnation 

5.  Shared  situational  awareness 

6.  Self-synchronization 

7.  Sustainability  and  speed  of  command 

Second,  we  live  in  a  world  that  is  more 
tightly  coupled  than  ever  before,  creating 
opportunities  and  challenges  for  direct  and 
indirect,  desirable  and  undesirable  effects. 

NCO  encompasses  Networked 
Enabled  Capability  (UK,  Australia),  Net- 
worked-Based  Defense  (Sweden),  and 
concepts  from  other  nations  also  based 
upon  operationalizing  the  tenets  of  Net¬ 
work  Centric  Warfare. 

Goals  and  Objectives 

Many  organizations  are  attempting  to 
analyze,  understand,  and  employ  Network 
Centric  Operations.  The  analytic  commu¬ 
nity  needs  to  support  the  military  services, 
joint  community,  and  other  stakeholders 
with  assessments  as  NCO  is  being  devel¬ 
oped.  The  community  should  play  a  lead¬ 
ership  role  in  creating  and  refining  the 
metrics,  processes,  methodologies,  models 
and  simulations  necessary  to  understand 
this  emerging  area.  The  community 
should  share  efforts,  successes  and  failures 
in  the  key  capabilities.  As  a  first  step,  the 
goal  of  the  meeting  was  to  provide  an 
opportunity  to  bring  a  multi-disciplined 
team  of  analysts,  operators  and  engineers 
from  those  organizations  together  to  share 
their  work,  develop  a  common  view  of  the 
state  of  practice,  expose  members  of  the 
broader  analytic  community  to  their 
needs,  identify  shortfalls  and  potential 
solutions. 

The  overall  objectives  of  the  workshop 
were  to  provide  an  assessment  and  a 
roadmap  to  revitalize  the  state  of  the  ana¬ 
lytical  practice  as  it  relates  to  NCO,  and  to 
recommend  priorities  for  any  initiatives 
identified.  In  other  words,  to  assess: 

•  “State  of  Health”  -  Provide  an  assess¬ 
ment  of  the  state  of  analytical  practice 
related  to  NCO. 

•  “Findings”  -  Provide  a  roadmap  to  revi¬ 
talize  that  state. 

•  “Recommendations”  -  Recommend 
priorities  for  any  initiatives  identified  in 
the  roadmap. 
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Figure  1:  NCO  Conceptual  Framework 


Some  specific  objectives  for  the  Working 

Groups  were  as  follows: 

•  WG  1:  Measures  of  Merit.  A  great 
deal  of  work  has  gone  into  developing  a 
conceptual  framework  for  Network 
Centric  Warfare.  An  extensive  body  of 
literature  exists  and  robust  discussion 
continues.  Measures  of  merit  have  been 
addressed  often  in  this  context.  Several 
approaches  have  been  proposed  for 
measuring  the  effectiveness  of  Network 
Centric  Operations.  Much  of  the  dis¬ 
cussion  has  focused  on  measures  of  per¬ 
formance  for  the  network  that  enables 
NCO.  One  reason  for  this  inclination  is 
that,  relative  to  other  areas  of  NCO  the¬ 
ory,  data  is  readily  available.  However, 
NCW  is  not  simply  a  new  communica¬ 
tions  system  with  embedded  decision 
support.  Therefore,  it  is  important  to 
examine  the  impact  of  NCW  technolo¬ 
gies  and  practices  across  the  physical, 
information,  cognitive  and  social 
domains.  To  devise  metrics  for  these 
other  aspects  of  NCO,  several 
approaches  were  considered  for  the 
workshop.  The  Network  Centric  Oper¬ 
ations  Conceptual  Framework  (Figure 
I)  was  the  starting  place.  Discussions 
on  cognitive  and  social  aspects  also 
were  reviewed  to  ensure  that  the  appro¬ 
priate  measures  and  metrics  were  con¬ 
sidered. 

•  WG  2:  Processes  and  Methodologies. 
There  were  three  objectives:  (1)  use  the 
NCO  Conceptual  Framework  to  assess 
operations  analysis  ability  to  evaluate 
NCO  enabling  capabilities,  (2)  identify 
areas  of  weakness  in  both  the  NCO 
Conceptual  Framework  and  the  OA 
Process,  and  (3)  recommend  near  and 
longer  term  actions  that  will  improve 
the  operations  analysis  community  to 
.support  NCO  related  research,  analyses, 
and  implementation. 

•  WG  3:  Assessment  Tools.  The  objec¬ 
tive  of  this  working  group  was  to  assess 
various  models  across  the  four  Network 
Centric  Warfare  domains.  Models  are  a 
set  of  algorithms  and  data,  with  support¬ 
ing  infrastructure  that  manages  output, 
random  number  generation,  etc.  For 
each  major  algorithm  that  models  an 
important  aspect  of  NCO,  whether  con¬ 
tained  in  the  existing  tools  or  those 
under  development,  the  modeling 
process  was  examined  to  determine  if  it 
is  mature,  developing,  immature  or 


unknown.  The  working  group  also  dis¬ 
cussed  if  research  is  needed  to  increase 
understanding  of  how  specific  NCO 
aspects  can  be  assessed  or  new  algo¬ 
rithms  developed.  For  data,  an  insight 
gained  from  a  model  is  the  combination 
of  the  data  and  the  model’s  transforma¬ 
tions  of  input  data  into  output.  Given  all 
of  the  existing  and  proposed  tools  that 
may  be  used  for  NCO,  the  state  of  the 
data  required  to  make  these  tools  “study 
ready”  was  assessed. 

WG  4:  NCO  and  the  Force  Transfor¬ 
mation  Process.  Working  Group  4 
examined  how  operations  research 
methods  can  be  applied  to  NCO  in  the 
context  of  the  Force  Transformation 
Process.  Transformation  will  only  hap¬ 
pen  when  eight  “key  elements”  co¬ 
evolve  to  take  full  advantage  of  infor¬ 
mation  age  technologies,  capabilities 
and  opportunities.  These  eight  key  ele¬ 
ments  are:  (1)  concepts;  (2)  leadership; 
(3)  education;  (4)  culture;  (5)  training; 
(6)  organization;  (7)  process;  and,  (8) 
technology.  The  principal  task  for  the 
Force  Transformation  group  was  to 
understand  how  OA  techniques  can  be 
applied  across  the  eight  “key  elements” 
of  transformation  and  balance  the  trans¬ 
formation  process. 

WG  5:  NCO  and  Coalition  Forces. 
There  were  three  major  objectives  of 


this  working  group:  (1)  to  review  and 
discuss  papers  that  address  current  prac¬ 
tice  in  the  areas  of  tools,  studies,  and 
experimentation  or  live  events;  (2)  to 
discuss  each  individual  country’s  state  of 
practice  of  OA  in  support  of  NCO,  to 
identify  good  and  bad  points  and  to  iden¬ 
tify  future  directions;  and,  (3)  to  assess 
collaborative  efforts  among  coalition 
partners  to  detennine  shortcomings  and 
to  recommend  ways  to  improve  collabo¬ 
rative  OA  efforts.  Papers  were  presented 
in  three  areas:  (1)  country  study 
overviews;  (2)  tools;  and,  (3)  experi¬ 
ments  and  live  events. 

•  WG  6:  Applying  NCO  to  an  Actual 
Event.  This  working  group  examined 
the  application  of  Network  Centric 
Operations  in  support  of  recent  opera¬ 
tions  in  Afghanistan  and  Iraq.  The  work¬ 
ing  group  reviewed  the  data  for  Opera¬ 
tions  Desert  Storm  and  Iraqi  Freedom  to 
look  for  the  limiting  factors,  the  tools 
and  techniques,  and  the  different  ways 
that  operations  analysis  can  help  take 
information  and  enable  events. 

State  of  the  Practice 

The  working  groups  were  asked,  if  pos¬ 
sible,  to  provide  their  “State  of  Health”  in 
two  contexts:  the  Operations  Analysis 
Process  (Figure  2)  and  the  Network  Cen- 

(SeeNCO.p.32) 
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Figure  2:  Operations  Analysis  Process 
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(continued from  p.  31) 

trie  Warfare  four  key  domains  (Figure  3). 
Additionally,  the  seven  tenets  of  NCO  in 
relation  to  the  framework  were  evaluated 
by  the  Processes  and  Methodologies 
Working  Group  and  the  eight  key  ele¬ 
ments  of  transfonnation  were  considered 
by  the  Force  Transformation  Working 
Group. 

Within  the  OA  Process,  the  measures 
were  evaluated  as  frequently  being  used, 
but  inconsistent  in  their  application.  In 
addition,  the  immaturity  of  an  “NCO  OA 
Process’*  was  considered  consistent  with 
the  state  of  the  understanding  of  NCO, 
and,  in  the  context  of  NCO,  the  tools  were 
used  infrequently.  Current  measures, 
tools  and  practices  are  understandably 
grounded  in  what  we  know  and  the  deci¬ 
sion  process  we  support,  but  applying 
operations  analysis  in  the  area  of  Network 
Centric  Operations  is  in  its  infancy.  The 
US  Amy’s  Future  Combat  System  and 
the  Navy’s  FORCEnet  activities  are  initial 
efforts. 

In  applying  operations  analysis  to 
NCO  events,  discussions  highlighted  the 
perceptions  that:  the  OA  process  is  cur¬ 
rently  limited  by  available  and  valid  data, 
the  real-world  operation  is  a  dynamic 
environment  and  the  data  is  not  clean  or 
consistent,  warfighting  systems  do  not 
output  analytically-ready  data,  and  data 
collection  must  be  pre-planned  due  to  the 
numerous  organizations  involved  and 
their  focus  on  the  real-world  operation. 

The  state  of  health  for  OA,  in  relation 


to  the  NCO  four  key  domains  shown  in 
Figure  3,  was  estimated  as  deficient  or 
poor.  Even  at  the  physical  level  of  model¬ 
ing  and  analysis,  the  Operations  Analysis 
community  continues  to  focus  on  the  war¬ 
fare  (combat)  portion  of  NCO  and  not  on 
the  larger  set  of  military  missions  (e.g., 
OOTW)  -  as  observed  in  the  scarcity  of 
methods  for  these  situations.  In  general, 
applying  OA  across  the  four  domains  was 
considered  fair  or  poor:  physical  -  fair; 
information  -  fair;  cognitive  -  poor;  and, 
social  -  poor.  Overall,  the  OA  community 
seems  to  be  uncomfortable  with  expanding 


its  analysis  into  the  cognitive  and  social 
levels.  Scientific  approaches  of  the  past 
that  addressed  the  physical  domain  well 
apparently  do  not  represent  the  other 
domains  as  well,  and  usually  are  limited  in 
focus  to  key  physical  interactions  associat¬ 
ed  with  combat  and  combat  support  opera¬ 
tions.  Methods  from  other  disciplines, 
such  as  medicine,  genetics,  sociology,  psy¬ 
chology,  political  science  and  cultural 
anthropology  fields,  should  be  considered 
as  potential  sources  of  knowledge  that 
could  inform  the  OA  process  and  commu¬ 
nity. 


Physical  Domain 

where  strike,  protect,  and  maneuver  take  place  across 
different  environments 
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Figure  3:  New  Four  Key  Domains 


Figure  4:  Overall  Assessment  of  Applying  OA  to  NCO 
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Over  five  years  ago  at  the  MORS  Mini- 
Symposium  and  Workshop  on  ‘‘Analyzing 
C4ISR  for  2010,”  a  template  for  the 
NATO  Code  of  Best  Practice  for  Assess¬ 
ing  Command  and  Control  was  developed 
and  evaluated.  This  template  was  used  to 
evaluate  the  OA  process  for  NCO  and  the 
Synthesis  Group’s  overall  assessment  in 
having  examined  assessments  from  all  six 
working  groups  is  contained  in  Figure  4. 
There  seems  to  be  an  overall  decrease  in 
the  adequacy  of  the  process,  due  mainly  to 
a  feeling  that  NCO  as  a  whole  may  be 
more  difficult  to  evaluate  then  C4ISR 
alone. 

With  inputs  from  each  of  the  working 
groups,  the  Synthesis  Group’s  overall  qual¬ 
itative  assessment  across  the  NCW  four 
key  domains  is  provided  in  Figure  5.  The 
physical  domain  is  assessed  to  be  in  better 
“health”  from  the  application  of  the  OA 
process  than  the  other  three  domains. 


Figure  5:  Qualitative  Assessment  Across  NCW  Four  Key  Domains 


Synthesis  Group  Findings 

After  participation  among  the  working 
groups  and  detailed  discussions  in  their 
meetings,  the  Synthesis  Group  submitted 
their  fmdings. 

Working  Group  1:  Measures  of  Merit 
(MoMs)  -  The  findings  focused  on  the 
assessment  that  the  MoMs  for  NCO,  espe¬ 
cially  for  understanding  a  networked 
“cause  and  effect”  relationship,  are  a  work- 
in-progress.  However,  previous  MoMs  for 
the  physical  domains  for  warfighting  mis¬ 
sions  are  well  understood  and  remain  rele¬ 
vant. 

•  MoMs  must  both  enable  the  analyst  and 
inform  the  client;  these  may  be  starkly 
different  for  Network  Centric  Operations 
Analysis  (NCOA)  than  for  traditional 
OA.  MoMs  for  NCOA  are  a  work-in- 
progress, 

•  Survivability-related  MoMs  are  increas¬ 
ingly  relevant  to  military  operations  and 
remain  so  for  NCOA  as  well  (at  all  lev¬ 
els,  i.e.,  MOPs,  MOEs), 

•  A  combination  of  MoMs  is  necessaiy  for 
OA  of  networked  “cause-and-effect” 
relationships. 

Working  Group  2:  Processes  and 
Methodologies  -  The  findings  focused  on 
the  NCW  conceptual  framework  and  the 
gap  between  sense-making  and  decision 
making.  It  also  was  suggested  that  the 
relationship  between  cognitive  modeling 


and  sense-making  modeling  needs  to  be 
addressed. 

•  The  NCO  framework  emphasizes 
“Quality  of  Individual  Sense-making” 
and  the  “Quality  of  Group  Sense-mak¬ 
ing”  versus  decision  making. 

•  Many  in  the  NCO  community  assert 
that  cognitive  modeling  is  an  essential 
part  of  the  solution  to  modeling  sense- 
making.- 

•  Tlie  lack  of  processes  and  methodolo¬ 
gies  for  OA  using  sense-making  and 
cognitive  modeling  make  the  NCO 
framework  a  challenge  to  operational¬ 
ize. 

Working  Group  3:  Assessment  Tools  - 
The  findings  were  along  three  dimensions: 
“better”  data,  improvement  in  NCO 
assessment  tools,  and  the  networking  of 
assessment  tools  to  achieve  synergy  in  the 
application  of  a  tool  set. 

•  Some  tools  exist,  but  the  data  domain 
(both  real  blue  and  red  data)  is  still  too 
stove-piped. 

•  The  methods  to  change  information  to 
knowledge  seem  to  exist  within  the 
models,  but  are  not  well  understood. 

•  Networking  assessment  tools  is  neces¬ 
sary  but  not  sufficient;  we  must  learn 
to  “do  better  things,”  not  “do  things 


better.” 

•  The  networked  force  must  be  treated  as 
a  holistic  entity  not  as  a  sum  of  discrete 
elements. 

Working  Group  4:  NCO  and  Force 
Transformation  -  To  effect  Force  Trans¬ 
formation  it  was  concluded  that  the  further 
we  move  away  from  assessing  transfor¬ 
mations  that  are  focused  on  a  single  Ser¬ 
vice,  tactics,  and  material  (  “old”  mis¬ 
sions,  etc,),  the  fewer  OA  tools  are 
available  to  the  analyst  for  information, 
cognitive  and  social  domain  related 
assessments. 

Working  Group  5:  NCO  and  Coalition 
Forces  -  It  was  discussed  that  the  OA 
timelines  are  not  fast  enough  to  keep  pace 
with  technology  and  other  developments. 
The  experience  of  the  UK’s  embedded 
OAs  needs  to  be  assessed  and  their  exam¬ 
ple,  maybe,  followed.  However,  the 
inclusion  of  civilian  agencies  also  must  be 
addressed. 

•  OA  analysis  timelines  are  not  fast 
enough  to  meet  the  pace  of  technology 
and  DOTMLPF  development. 

•  OA  is  overly  focused  on  military  to  the 
exclusion  of  civilian  agencies. 

(See  NCO,  p.  34) 
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•  Need  to  consider  new  level  of  embed¬ 
ded  analysis  (OA  in-theater  with  data 
mining  and  assessment  tools)  to  provide 
real  time  feedback  on  NCO  effective¬ 
ness. 

Working  Group  6:  Applying  OA  to 
Actual  NCO  Events  -*  It  was  found, 
because  policies  and  doctrine  to  incorpo¬ 
rate  analysis  is  lacking  along  with  the  non¬ 
use  of  embedded  analysis,  that  OA 
becomes  an  afterthought.  If  these  findings 
and  the  data  domain  are  improved,  then 
the  assessment  of  actual  NCO  events  will 
be  improved. 

•  Lack  ofjoint  and  some  Service  doctrine 
to  incorporate  analysis  in  a  JTF  limits 
OA  support  of  NCO. 

•  Reliance  on  non-embedded  analysts 
makes  OA  an  afterthought  in  JTFs. 

•  OA  currently  limited  by  available  and 
valid  data 

•  Time  and  data  consti*aints  limit  the  tool 
set 

Extent  of  Collaboration 

As  with  the  State  of  the  Practice,  each 
of  the  working  groups  were  asked  to  pro¬ 
vide  a  self-assessment  of  the  level  of  col¬ 
laboration  among  the  Services,  Coalition 
Forces  and  US  Agencies  where  operations 
analysis  has  been  applied  to  NCO. 

The  level  of  collaboration  among  (and 
within)  the  Services  was  assessed  as 
inconsistent.  Collaboration  between  ser¬ 
vices  is  varied  and  has  two  dimensions: 
between  operators  and  between  analysts. 
Collaboration  between  analysts  was  good 
with  the  Joint  Lessons  Learned  Team 
because  they  were  located  at  all  Combat¬ 
ant  and  Component  Headquarters  -  so 
contact  was  physical.  Collaboration 
between  Service  teams  at  disparate  sites 
was  very  infrequent.  Most  were  com¬ 
pletely  unaware  of  the  others’  existence. 

The  collaboration  with  US  Agencies 
was  difficult  to  assess  because  the  majori¬ 
ty  of  the  attendees  did  not  know  of  exam¬ 
ples.  Working  Group  6  discussed  that  the 
OIF  Phase  IV  planning  involved  almost 
every  major  agency  in  the  US  govern¬ 
ment;  however,  none  of  these  agencies 
had  the  collaborative  capabilities  and 
expertise  to  contribute  effectively  and  in  a 
timely  manner. 
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For  the  Coalition  Forces,  participants  con¬ 
sidered  collaboration  to  be  infrequent  and 
inconsistent.  A  spectrum  of  change  of  mil¬ 
itary  forces  in  the  international  community 
exists  from  status  quo  to  modernization 
through  transformation  of  forces.  Nations 
are  at  different  points  in  the  spectrum  with 
respect  to  NCO,  but  ail  are  committed  to 
moving  away  from  the  status  quo  end  of 
the  spectrum.  The  participation  in  various 
multinational  forums  is  evidence  of  the 
commitment.  It  was  noted  that  most  col¬ 
laboration  with  Coalition  forces  is  on  a 
bilateral  basis.  Many  of  the  activities  are 
modest  events  between  coalition  partners 
(e.g.  entropy  work,  agent  based  modeling, 
S&T  assessment,  an  assessment  of  deci¬ 
sion  making  in  collaborative  environments, 
development  of  a  conceptual  model,  and 
working  groups  to  discuss  interoperabili¬ 
ty).  The  conclusion  is  that  there  is  still  a 
need  for  significant  collaborative  analytical 
events  between  Coalition  partners. 

Recommendations 

First,  the  Synthesis  Group  developed  a 
set  of  actionable  recommendations  from 
the  working  group  discussions  and  their 
findings. 

•  A  MORS  Symposium  working  group  or 
composite  group  dedicated  to  NCO  is 
needed.  NCO  is  much  more  than  C4ISR 
and  Information  Operations.  This  new 
working  group  or  composite  group  could 
be  charged  to  examine  the  OA  toolkit 
and  processes  to  better  focus  on  the 
NCO  problem.  At  the  72nd  MORSS  it 
is  proposed  that  the  CG  B  Chair  discuss 
this  issue  with  the  leadership  from 
Working  Groups  5-10,  providing  recom¬ 
mendations  to  the  MORS  Working 
Group/Composite  Group  Committee 
Chair  and  the  Vice  President  (Meeting 
Operations).  [Ed  Note:  This  proposal 
will  be  addressed  during  the  72nd 
MORSS] 

•  Create  center(s)  of  excellence  within  ser¬ 
vices  and  JFCOM  for  NCO  assessments 
to  capture  and  share  results  and  data  of 
experiments,  training,  analysis  and  expe¬ 
rience. 

•  Enable  (e.g.  thru  agreements,  business 
practices)  the  networking  of  analysts 
working  NCO  and  provide  reach-back 
for  tools,  data  and  previous  studies. 

•  Establish  a  “business  model”  that 
enables  services  to  conduct  operations 
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analysis  in  support  of  “born  joint”  NCO 
concepts  and  that  is  underpinned  by  a 
set  of  approved  tools  and  data. 

•  Conduct  follow-on  MORS  special 
meetings  to  share  NCO  analysis  and 
experimentation  results.  There  were 
two  options  recommended: 

~  A  2-day  or  3-day  Mini-Symposium  to 
educate  the  community  in  the  area  of 
NCO  analyses.  Training  and  educa¬ 
tion  was  seen  to  be  critically  impor¬ 
tant  to  Transformation  overall,  and 
specifically,  to  the  operations  analysis 
practice  in  support  of  NCO. 

-  A  second  3-day  Workshop  to  discuss 
the  issues  more  fully.  It  was  suggest-, 
ed  to  provide  foundation  presenta¬ 
tions  during  the  morning  of  the  1st 
day  and  outbriefs  during  the  after¬ 
noon  of  the  3rd  day  -  this  would 
leave  two  full  days  for  discussion  and 
further  debate.  During  this  initial 
workshop,  the  working  groups  felt 
they  only  scratched  the  surface  in 
their  discussions. 

Synthesis  Group  Summary 

Tlie  Synthesis  Group  summarized  their 
findings  thus: 

The  good  news  is  that  the  OA  process 
developed  in  the  Industrial  Age  has  con¬ 
tinued  applicability  to  Network  Centric 
Operations  (NCO)  in  the  Information  Age. 
We  are  only  beginning  to  understand  the 
practices  that  should  be  followed  to  per¬ 
form  credible  NCO  analyses.  In  addition, 
we  know  there  are  NCO  considerations 
that  remain  challenges.  The  bad  news  is 
that  we  do  not  do  a  satisfactory  job  in  doc¬ 
umenting,  sharing,  and  learning  from 
NCO  analyses  perfonned. 

The  worse  news  is  that,  with  the  arrival 
of  terrorism,  other  asymmetric  threats, 
rogue  nations,  and  increased  concern  for 
homeland  defense,  the  challenges  of  effec¬ 
tively  assessing  NCO  and  efficiently 
accomplishing  the  goal  of  Force  Transfor¬ 
mation  are  complex  and  urgent.  Although 
these  are  daunting  challenges,  there  is 
some  better  news  that  is  implicit  in  this 
workshop  itself.  If  the  recommendations 
of  this  workshop  are  implemented  suc¬ 
cessfully,  they  will  provide  a  basis  for 
improving  the  operations  analysis  process 
and  empowering  the  OA  community  to 
address  the  most  critical  analytical  chal¬ 
lenges  to  support  NCO  and  Force  Trans¬ 
formation.  O 
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